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ABSTRACT 


The problems of estimation and control of discrete, linear, time- 
varying systems are considered. Previous solutions to these problems in- 

• t 

volved either approximate techniques, open-loop control solutions, or 
results which required excessive computation. 

i * 

The estimation problem is solved by two different methods, both 
of which yield the identical algorithm for determining the optimal filter. 
This algorithm is the partitioned equivalent to a solution suggested in 
1964 by formulating the time delay system in an "expanded state" repre- 
sentation. The partitioned results achieve a substantial reduction in 
computation time and storage requirements over the expanded solution, 
however. The results reduce to the Kalman filter when no delays are pre- 
sent in the system. 

The control problem is also solved by two different methods, 
both of which yield identical algorithms for determining the optimal 
control gains. This result is also a partitioned solution to the "ex- 
panded state" representation of time delay systems and also achieves 
savings of computation time and storage requirements. The stochastic 
control is shown to be identical to the deterministic control, thus ex- 
tending the sepai*ation principle to time delay systems. The results 
obtained reduce to the familiar optimal control solution when no time 
delays are present in the system. 



I 

§ 

t 

. > 

* * 

A 

The principle of duality of estimation and control is shown to 

•*. 

he extended to time delay systems. 

Necessary and sufficient conditions are developed r for the obser- 

* 

vability and controllability of discrete linear systems with time delay. 
An exhaustive bibliography of publications dealing with optimal 

* 

estimation and control of time delay systems over Lbi period 1960-1970 
is included. 


CHAPTER, l 


INTRODUCTION / 

1.1 Introduction, 

* 

Thn study of systems with time delay Is not a new one. During 
the past decade (1960-1970) over 200 papers have appeared dealing with 
the optimal control and estimation of linear systems with time delay* 
Closed-form solutions are difficult to obtain because the analytic ex- 
pression for a time delay differs from the form used to describe the 
rest of the system. For example, in the frequency domain, the system 

t . 

is usually described by a ratio of polynomials in s, but the time M'.ay 

■ -*Ts 

is expressed as the transcendental function, e * . This may be put in 

a common form by expressing e“ Ts as an infinite series in s. Unfortun- 

* f ... 

ateiy, this results in a system of infinite order with an, infinite num- 
ber of poles. Since the exact solution to such a system is computa- 

* 

tionally impossible, the normal procedure is to truncate the approxima- 
tion to the time delay. This results in a solution which is subop timal. 

In the time domain, the describing equations which are ordinary 
linear differential equations when no delay is present, become differen- 
tial-difference equations to describe the time-delay effect. The time 
delay may be represented by a differential equation of infinite order. 
This also results in a system of infinite order, however, so a truncated 
series is generally used, resulting again in a suboptimal solution, - 


1 


2 


■p' 


* 


R 


A third way of describing the system is in discrete time. This 

is naturally suited for time delay representation because the system and 

* * #. 

titoe delay can be described as difference equations. A major problem 

* 

exists in such, a representation because the resulting system is directly 
proportional to the magnitude of the longest time delay. Once again the 

a 

computation can quickly become excessive. 

» , 

The. approach of this paper is further exploration of the discrete- 
time representation of systems with time delay*. Substantial savings in 
both computation time and storage requirements are achieved over previous 

discrete-time solutions to systems with time delay. 

♦ * * 

1.2 Previous Work 

An extensive bibliography is presented in Appendix C which con- 
tains a list of more than 200 papers published in the last decade, which 
deal with time delay systems. In general, these pipers fall into three 
categories which limit the practicality of their implementation. 

1. The majority of the control results are open-loop. This is 
of little value for stochastic systems where plant disturbances occur. 

2. The computational requirements of the proposed solutions are 
too great for practical implementation. 

3. Approximate methods are used, resulting in suboptimal solu- 
tions. 

The previous work, pertaining to this dissertation is discussed in detail 
in Chapters 2 and 3. 
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1.3 Organjzation 

Chapter 2 develops an algorithm, for optimal estimation of dis- 
crete linear systems with, time delay using properties of conditional ex- 
pectation. This result is shown to reduce to the Kalman filter when no 
time delay is present. 

Chapter 3 develops an algorithm for optimal control of discrete 
linear systems with time delay using dynamic programming. The stochastic 
control is shown to be the same as the deterministic control for time 
delay systems, thus extending the separation principle to such systems. 

Both results are then shown to reduce to the familiar optimal control 

a * 

solutions when no time delay is present. 

Chapter 4 'introduces the "expanded state" form, an alternate re- 
presentation of time delay systems in discrete time. This new form is 
then used to obtain results identical to those of Chapters 2 and 3. The 
expanded state representation of time delay systems is not unique to this 
paper. It has been examined by previous 'authors and found to be compu- 
tationally unattractive because of the resulting increase in system or- 
• * 

der which is proportional to the magnitude of the longest time delay. 
Chapter 4 results in a partitioned algorithm solution whereas previous 
authors retained the entire expanded matrices in their original form. 

Analytical expressions are developed in Chapter 5 for the com- 
putational and storage requirements of the partitioned solutions and 
compared with the requirements of the expanded state representation. 
Although identical results are obtained to the estimation, and control 


problems, the partitioned solutions of; this paper result in a 30% to 
90% reduction in computation time, and a 30% to 60% reduction in storage 
.requirements over the expanded form. 

Chapter 6 summarizes the results and suggests future areas of 
study for time delay systems. '.f 


CHAPTER IT. 


OPTIMAL ESTIMATION IN LINEAR DISCRETE SYSTEMS 
WITH. TIME DELAY 

2.1 Introduction 

In this chapter, the* problem of estimation is examined for sto- 
chastic linear discrete systems with, time delay. It is assumed that 

B- 

both, plant and measurement noise are present. An algorithm is obtained • 
for estimating the state of the system. 

Section 2.1 formulates the general estimation problem for dis- 
crete systems., whether time delay is present or not. The familiar dis- 
crete-time system model is, modified to incorporate the effects of de- 
layed state values into the system behavior. 

In Section 2.3, previous work, on stochastic time delay systems 
is reviewed. One of these works, that of Priemer and Vacroux,^" obtains 
a similar result to that obtained in Section 2.6. The result in this 
chapter is achieved using properties of conditional expectation whereas 
the referenced work uses orthogonal projection. This result also in- 
cludes systems with a control input whereas the work of Priemer and 
Vacroux does not. 

Section 2.4 establishes basic results in estimation theory which' 
are necessary for the results of Sections 2.5 and 2.6. In some instances 
the proofs accompany these basic results. In others where the proofs 
are considered unnecessary or burdensome, the reader is referred to the 
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literature. 

Section 2.5 uses the properties of; conditional expectation to 

* 

develop an 'expression for the one-step prediction process., 

* 

Properties of conditional expectation are again used in Section 

2.6 to develop the estimation algorithm for stochastic discrete-time 

* * * * 

systems with, time delay. This- result is demonstrated to reduce to the 

familiar Kalman filter when no delay* is present in the system. 

Section 2.7 summarizes the results, of this chapter and discusses 
the computational aspects of the estimation algorithm obtained. 

2.2 Optimal Estimation for Discrete Systems 

In this section the general problem of estimation. for discrete 
systems is considered. That i's, the state and measurement processes of 
a dynamic system are discrete-time stochastic processes. 

Consider a dynamic system S whose state as a function of time 
is an n-dimensional discrete— time stochastic process {x(.k) , kel} where 
either I « {k:k - 0, 1, ... , N} or I - {k:k - 0, 1, 2, ... } . Sup- 
pose that it is desired to know the value of x(k) for some fixed k, 
but that x(k) is not directly accessible for observation. In addition, 
suppose that a sequence of measurements z(l), ... , z(j) are available 
which are causally related to x(k) by means of some measurement system 
M as shown in Fig. 2.1^ and it is desired to utilize these data to infer 
the value of x(k). Let £z(i), i = 1,2, , j} be an m-dimensional , 
discrete-time stochastic process. 

. Since only the measurements zCl)> ... , .z(j) are available from 
which to estimate x(k) , let the estimate of x(k) based on these 





Figure 2.1 Block Diagram of Dynamic System, S, With State x(k) and 
Measurement System, M, With Measurements z(i) 



measurements, be denoted by x(k| j) and define it to be some n-dimensional 

• * 

vector-valued function f^ of the measurements! 


x(k|j) « f. iz(i), i « 1, ... , j) 


The estimation problem is, one of determining f^ in some rational 

k 

and meaningful manner. The approach, to be used in this chapter is based 


on consideration of the estimation error, x(k|j), which, is defined by 
the relation , 


xCk-U) = x(k) £(k|;j) (2.1) 

Ideally, x(k}j) = 0 and the estimate is exact. When x(k|j) £ 0 
a penalty is assigned for the incorrect estimate. This is done by 
specifying a penalty or loss function L = L{x(k|j)} which has the follow 
ing properties: 

1. L is a scalar-valued function of the n variables 

2. L(0) = 0 where 0 denotes the null n-vector 

— .«* 

3. Ltx b (.k|j)J < L[x a (kJ j )] whenever p[x b (k|j)] <p[x a (k|j)] 
where p is a scalar-valued, non-negative, convex function 
of the n-variables - 

4. LlxCkjj)] - L[-x(k| j ) j • ' 

The first property is essential to the obtaining of a unique 
minimum associated with a family of vectors. 

The second property simply specifies that there is no penalty 
when the estimate is exact. . 

In the third property, p is a measure of the distance of x(k| j) 
from the origin in n-dimensional euclidean space, and L is specified to 


9 


be a non-decreasing function of this distance. That is, as x(k|j) be- 
comes “closer" to zero, the penalty decreases. 

\ 

The* fourth, property requires that L[*j be symmetric about the 

/ 

origin. 

A loss function that possesses, the above four properties is 

termed an admissible loss function . It should be noted that L need not 

* ♦: 

be a convex function. 

Since x(k) and x(k| j) are random vectors, it follows that x(k| j) 
is also a random vector and that L is a random variable. In order to 
obtain a useful measure of the loss, a performance measure J is defined 
as the mean value of L. 

The familiar discrete-time model for a stochastic process may be 
expressed as 

x(kd*l) « <{> (k+l,k)x(k) + ^ (k+l,k)u(k) . + p (k+l,k)w(k) (2.5) 

z(k+l) = H(k+l)x(k-KL) + v(k+l) (2.6) 

where x = n vector (state) 

u - r vector (control) . ' 

w = p vector (plant disturbance) 
z - m vector (measurement) 
v — m vector (measurement disturbance) 

<{) => n x n (state transition matrix) 

p - n x p (disturbance transition matrix) • 

. i(i a n x r (control transition matrix) , 

H. = m x n (measurement matrix) 



(w(k), k « 0, X, . . .} ^ zero mean gaussian white sequence having a posi- 

¥ 

tive semi -definite p x p covariance matrix {Q(k) t k *= 0, 1, ... }. That 
is , . 

! 

* • 

*% 

£{w(k)} « 0, k » 0, 1, ... . (2.7) 

fclwCjVCk)} - q(k)s k (2.8) 

* . 

where is. the Kronecker delta. 

, fc(k+l), k=0 , 1 , . . . } is a zero mean gaussian white sequence having a posi 
tive semi-definite m x m covariance matrix (R(k-KL) , k=0,l ,...}. 

t{v(k+ 1)} = 0, k = 0, 1, ... C2.9) 

, * 

E{v(j+l)v'(k+l)} - R(k+l)6^ k (,2.10) 

Attention is restricted to the case where the two stochastic processes 
'{WOO, k = 0, 1, ... } and‘{v(k+l), k- 0, 1, ...} (2.11) 

are independent of each other. That is 

E jv (j)w’ (k) } = 0 for all j = 1, 2, ... , k = 0, 1, ... (2.12) 

The initial state x(0,) is a gaussian random n-vector with zero mean, 
having an n x n positive semi -definite covariance matrix 

Eix(.0)x’(0)}> P(0) (2.13) 

It is assumed that x(0) is independent of (w(k) , k - 0* 1, ...} and 
{v(k+l), k « 0, 1, . . . } so that 

1, 


E £c(.0)w’ (k) } = 0, k s 0, 


• • • 


(2.14) 


I 

i 
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E{x(0)v’ (k+1)} « 0, k B 0, 1, (2,15) 

» 

It is also assumed that the cdntrol sequence Is either known or may be 
specified. 

Expression (2.5) states that the value of the state vector at 
the* next Instant of time depends only on the present values of the 
state, control, and plant noise vectors. If, however, this future value 
of the state vector is also dependent upon past values of the state vec- 
tor^ then Eq. (2.5) must be modified to indicate this dependence. This 
is done by re-writing Eq. (2.5) as 

J 

, x(k+l) ** ({. i (k-M,k)xCk,-i) + ij; (k-+l,k)u(k) + p(k+l,k)w(k) (2.16) 

The index J in Eq. (2.16) indicates the most distant (in time) value of 
state vector, x(k-J), that affects the state vector at the next instant 
in time. It is recognized, of course, that many of the <^(k+l,k) may 
be zero, indicating that not all delayed states back to x(k-J) affect 
the expression for x(,k+l). Thus <j>^Ck+l,k) is the state transition matrix 
associated with the delayed state vector x(k-i) in Eq. (2.16). Proper- 
ties expressed by Eq. (2.13), (2.14) and (2.15) must also be modified 
accordingly, as below, to include systems with time delay. 

The initial and delayed initial state {x(-i), i - 0, 1, ... , J) 
is a gaussian random n-vector with mean 

E {x (— i) } - 0 , i “ 0 , 1 , . . . , J (2.17) 

having the n x n positive semidefinite covariance matrices 







a 




5 

» 
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ElxC-i)x'H» c P("i. -1) 1, j » 0, 1 J. 

* 


(2.18) 


It is assumed that' £x(-*i), i « 0,l,...,j} is independent of 

*♦ 

(w(k), k » 0, 1, ...} and (v(k+l), k « 0, 1, . . . } so that' 

fc{x(-i)w'(k)} = 0 i«0,l,*...,J k. »» 0, 1, ... (2.19) 

* * 

E£acC-i)v* (K+l)> * 0 - i « 0,1,..., J k « 0, 1, ... (2.20) 


In addition, it should be noted that the system described by 

Eq. (2.16) and Eq. (2.6) must be observable or the following theory has 

# 

little meaning. A system is ’’observable" if every state in the corres- 
ponding deterministic system can be exactly determined from measurements 

* 

of the output over a finite interval of time. The reader is referred 
to Appendix A for a discussion of observability of discrete linear sys- 
terns with, time delay. 

With the system model described by Eqs. (2.6) - (2.11) and Eqs. 


(2.16) — (2=20) , the specific estimation problem may now be formulated. 

. Given the state sequence fe(k), It - 0, 1, ...} and the measure- 


ment sequence fe(i), i - 1, 2, ... , determine the estimate, x(kjj), 
of x(k) such that the expected value of- the mean square filtering error 
is. minimized . That is, it is desired to minimize E fr.[x (k|j)i } where 


H X (k I j ) J = x.(k ! j ) x ’ (jkji; 
x(k |j) = x(k) - x (k j j ) 


( 2 . 21 ) 

( 2 . 22 ) 


\ 


. The remainder of this chapter is devoted to determining the op- 

•* 

tiraal filtered estimate {x(k|j), tevjK As a byproduct, however, e*~ 

pressions are also obtained for the one-step prediction estimate 

* * 

ix CK I j ) » k“j+l) and for a limited range of smoothing estimates 

IS<k|j), k«j-l, i-2, ... , j-J}. 

These estimates are all obtained for discrete linear systems 

» * 

with. time delays. 

2.3 . Review* of Previous Work, on Estimation of Time Delay Systems 

Despite the great number of papers 1 extending the original work 

2 * 
in 1960 of Kalman and Bucy on linear filtering, a number of years 

passed by before optimal filtering of systems with, time delays was dis- 
cussed. The bibliography contains an extensive list of such literature. 
The three articles which discuss linear systems with delay in the plant 
only (not in measurement or control) are discussed below. 

♦ 

In 1967, the theory developed by Kalman and Bucy was first ex- 

3 

tended to linear systems with, multiple time delays by Kwakernaak whose 
development is for continuous systems. The single-variance equation of 

t 

the Kalman-Bucy theory is replaced by a partial differential equation 
and three boundary conditions. The boundary conditions are also partial 
differential equations. No explicit solution to these equations appears 
possible in closed form and solutions are not feasible for on-line anal- 
ysis of discrete time systems. Thus, although, the solution is theoreti- 
cally presented, i.t is impossible to implement in practice due to the 
extensive computation required. Thi.s- is substantiated by the fact that 
no literature appears in which this method is implemented. 


14 


The problem of developing a filter that was computationally 

* 

more feasible and that could be used in practice was resolved in 1969 

by Priemer and Vacroux.^ They offered a solution for discrete linear 

* 

t 

* * 

systems with, time delay which, avoided the necessity of expanding the 

order of the system. The computation time was considerably reduced 

* 

3 * 

from that required by the method of Kwakernaak. Chapter 4 of 

tliis dissertation demonstrates that Priemer’ and Vacroux’s result is iden- 
tical to that obtained by expanding the state space. 


In addition to these papers is the method of approximating the • 
time delay by a Taylor’s Series expansion or by a Pade approximation. 
Unfortunately, the solution obtained is subop timal^ since a truncated 
approximation to the delay Is used. No work appears in the literature 

discussing this type of approximation in relation to filtering of time 

* 

delay systems,. 


Both, papers discussed above consider the filter problem only 


and do not provide a complete solution for optimal control of stochastic 
time-delay systems. In 1969, however, Larson and Wells^ were able to 
do so by restricting attention to serial systems . Although their solu- 
tion was optimal only for single-input, single-output systems with delay 

in the plant, it is very easy to implement and computationally 

faster than any of the methods discussed in the above paragraphs. Their 

paper combines optimum estimation, prediction and control. 

It is also recognized that if it can be shown that the separation 
principle applies to time delay systems ; then the optimal filter may be 
developed independent of the optimal control solution. This is, in fact, 

j _ 

demonstrated in Chapter 4 of this dissertation. 


- \ „ 



I 
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In summary, them, with one exception, the results shown thus far 
In the literature for estimating systems with, delays in the plant, are 
limited, as to practical application., by one or more of the following: 

1. Excessive computation time, 

2. Excessive computer storage requirements ( 

3. Approximate methods resulting in subop t final results 

The. one exception i.s the paper by Pri.emer and Vacroux. Their results 
are obtained by* tw.o alternate methods in this dissertation. Conditional 
expectation properties are used in this chapter and an expanded state- 
representation is used in Chapter A. In both, cases, the work of Priemer 
and Vacroux i.s extended to include control inputs, 

2. A . Fundamental Theory of Estimation 

In this section some fundamental results of discrete estimation 
theory are obtained. Although., in some cases, more general results may 
be established, only those which are necessary- to the development of 
Sections 2.5 and 2.6 are presented. The reader is referred- to the 
literature in those cases, where the presentation of the proof is felt 
to be burdensome .or unnecessary. 

The following properties of gaussian conditional expectation are 
of fundamental importance in the next two sections. Here x, y, z are 

5 

gaussian random vectors. For proofs the reader is referred to Meditch 
(pp. 92-103) . 

1. Ebc|y} is a gaussian random vector which is a linear com- 
bination of the elements of y. 


(2.23) 



2. x — E{x| y} i.s independent of the. random vector ohtained by 

any linear transformation on y. (2.24) 

3. If y and z are independent^ where z i,& a rand ora’ m-vector^ then 

E{x|y,z} «E{x|y) + E{x| z) -x (2.25) 

4. For y and z not necessarily independent, 

E(x|y,z} « E(xly,z} - E^x|y) + E(x|z) - x (2.26) 

where t -■ z - E{zjy). 

* 

The following theorem can now be proved. 


Theorem 2.1 . If the- loss function is defined as 
L = [x(.k) - R(k| j)J ! [x(k) -£(k|j)J and 
{x(k), kcl) and’ (z(i), i - 1, ... , j) are dis- 
crete time stochastic process.es. then 

£(k.|j) - E {x (k) | z * (j ) } (2.27) 

where z*(j) m z(0) 

z(l) 

• ' 

z(j) 

Proof: Recall from the problem statement, Eq. (2.21) and Eq. (2.22), 

that x(k|j) is to be chosen such that L is minimized. By taking the 
gradient of L in Eq. (2.21) with respect to x and setting it equal to 
zero 


V,L = -2E £(k) - 


(h| j) |z(!) , 


, z(j)} - 0 


3 ? 


• • 


(2.28) 


Taking the transpose of hoth, aides of Eq. (2.28) 


E{x(.k) |z(l) , ... , z(j)} - E{x(k| j) |z(l) /. . . , z(j)} (2.29) 


But the right-hand side of Eq. (2.29) la just 

Eft(k|j)|z*(j>> = «Ck|j) 

By substituting this result into’Eq. (2.29) 

» 

£ Ck J j ) ** E(x(k)|z(l), ... , z(4)} 
and the theorem is, proved.- 


(2.30) 


(2.31) 


Now: the final four properties necessary to the development of 
this chapter can be stated. The reader is again referred to Meditch** 
(pp. 92-103) for proofs. 

5. x(k,|j) and x(kjj) are gaussian random n-vectors 

Where x(k| j) =• x(k) - £ (k | j ) . (2.32) 

6. x(k|j) is. independent of any linear combination of the 

available measurements. • (2.33) 


7. x (k | j ) is unique. 

8. E{x|y}> x + p x/yJ.(y-y> 


(2.34) 

(2.35) 


where 


xy 

P 


yy 

X 


E {xy * ) 
E {yy * } 


- E(x> 


y - E {y} 


This concludes the results -necessary to develop the optimal 


filter. 


2.5 . Optimal Prediction for Discrete Linear Systems with Time Delay 

Although, the results of the preceding section are fundamental 
ones, they are of limited practical utility. Consider the gaussian case 
where the optimal estimate is given by Eq. (2.35) as 

*(k|j) = + p x (k)z*(j) p zA(j) z *Cj) lz * (j) " (2 ' 36: > 

where z*(j) * z(.0)„ 

2 ( 1 ) 


2 (j ) 


For each set of measurements, it is necessary- to compute the inverse of 

P a jm x jm matrix, where j is the number of measurements and 

z*(j)z*(j) 

m is the number ‘of elements, in the measurement vector. If m is 1 and 
there are 50 measurements, then a 50 x 50 matrix must be inverted. If 
it is des.ired to perform estimation "on-line", application of the above 
expression to generate the optimal estimate becomes impractical. 

What are desired, from a computational point of view, are 
efficient and practical algorithms for -processing the measurements se- 
quentially, hopefully in real time, to obtain a current estimate. All 
of the results stated thus far are independent of the system model. For 
this reason they are valid for the time delay case also. The remainder 
of this chapter is devoted to developing such algorithms for prediction 
and estimation of the states of a discrete linear system with time delay 


,4 
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* 

^•5. 1 .. Sy stem Model Properties 

The system model Is described by Eqv (.2.6) - (2.11) and Eq. ' 
(2.16) - (.2.20) where the fundamental system and measurement equations 
are 

J 

x(k+l) = £ <K (k+l,k)x(k-Ki.) + r(k+l,k)^(k)+i|j(.k+l,k)u(k) (2.37) 

1=0 

* * 

z(k+l) = H.(k+l)x(k+l) + v(k+l) (2.38) 

v *■ 

* 

This model has the following properties,. Proofs are In Appendix B,. 

1. The stochastic processes, (x(k), k = 0, 1, ... } and 


Xz(k), k = 1, 

... , j) are gauss ian. 

(2.39) 

2. 

E{x(j)w'(k)} = 0 for all k> j, j = 0, 1, 2, ... 

(2.40) 

3. 

E{z(4)w'(k)} = 0 for all k. £ j,’j =0, 1, 2, ... 

(2.41) 

4 f 

E{x(j)v*(k)} « 0 for all j and k. 

(2.42) 

5. 

EXz(j)v , (k)} = 0 for all k > j .- 

(2.43) 


■ t; 

It is helpful in the sequel to define the following error 

. / 

covariance matrices in terms of 


P(i ,m| k) = E{[x(i) - *(f|k.)] [x(m) *(rojk)J *■} (2.44) 

Case 1^ A, m>k; prediction Error Covariance Matrix (2.45) 
Case 2: i>k, m *= k; Prediction/Filtering Error Covariance Matrix (2.46) 
Case 3: £>k, m<k; Prediction/Smoothing Error Covariance Matrix (2.47) 
Case 4_: f = m = k; Filtering Error Covariance Matrix (2.48) 
Case .5: = k, m*k; Filtering/Sraoo thing Error Covariance Matrix (2.49) 
Case 6: % y m^c; Smoothing Error Covariance Matrix (2.50) 


if 


It should al$,o be clear from Ec^. (.2.44) that 


P(*,m|k) P* (m,£ |k) 


(2.51) 

where x(.i|k) *=* x(£) - x(i|k) 

» 

/ 

. 


■* prediction error if Z > k 


(2.52) 

«* filtering error if & » k 


(2.53) 

*= smoothing error if & < k 

. 

(2.54) 


2.5.2 . Optimal One -Step Prediction * ’ • 

The algorithm for the optimal one-step predicted estimate 
x(k+l|k) is developed below. Some important properties of the corres- 
ponding one-step prediction error x(k+ljk) = x(k+l) - x(k+l|k) are also 

»- 

established. Of particular interest is the nature of the stochastic 
process x(K+l|k), k = 0, 1, ... and the behavior of its corresponding 
covariance matrix 


. *<k+l,k+l|k) - E (x (k+1 { k)x ' (k+1 1 k) } ' (2.55) 

It is assumed that the optimum estimates {x(k-i | k) , i=0,i,...j} 
and the n x n covariance matrices {P(k-i,k-j | k) , i,j = 0, 1, ... , J} 
of the corresponding filtering and smoothing errors x(k-i) and x(k-j) 
are known for k = 0, 1, ... ; i,j - 0, 1, ... , J. The procedures for 
obtaining x(k-i.jk) and P(k-i.,k-j|k) are given in the next section. 

Prom the property given by Eq. (2.39) and Theorem 2.1 
x(j-i|j) - E 6c(j-i) | z (1) , ... , z(j) } (2.56) 

is the optimal estimate of x(j-i) for j = 1, 2, ... ; i = 0, 1, ... , J. 
For j = 0,' there are no measurements and it follows from Theorem 2.1 that 


ikC^llO) ** E {x G~i) | no measurements) 

* . * 

" E{x(rt)} 

■* 0 ■ . (2.57) 

It ts clear that i(j~i|j) Is gaussian from property (2.32). 

Similarly the filtering error 5tCj-l| j) . =* x(j) - £(j-i|j) is a zero mean 

gaussian random n-vector for which, it is assumed the covariance matrix 

j) is given. For j = 0, 

0 * 

*(-i|0) ■= x(.-i) - S(-i|0) 
m xC-t) - 0 

I 

so that P C-i., -i 1 0) - E {x C-iJ 0) x ’ C-i I o) 

« E (x(ri)x’ (rO 

« PC-i,-i) i, &, - 0,1,. . J 

• » 

where, the latter is assumed given In the system description. 

The following result can now be established for optimal pre- 

diction. 

■ * 

Theorem 2.2 If the optimal filtered estimate 
*tj-i|j) and the covariance matrix PCj-'i» | j) 
of the corresponding filtering error 
*Cj~i|j) ~ x(j-i) - ^Cj-i|i) are known for some 
j =0, 1 , ... ; i> i - 0, l s . . . , J, then for 
k = j + 1 

(a) The optimal predicted estimate &(k-KL[k) 
for all. admissible loss functions is given by the 


t 

(2.58) 


(2.59) 



/ 

1 * 

* * 

*> 

express ipa 

J 

x (k+1 1 k) * £ <{> . (k+1 , k) x (k-lj k) 

i *?0 

i 

+ i> (k+1 , k) u (k) 

Cb) The stochastic process {x(k+l|k), 

• • 

. k « 0, 1, 2, defined by the prediction 

* 

error relation 

* *• 

x (k+1 1 k) =. x(k+l) - x(k+l|k) 

i 

is a zero mean Gauss Markov-(J+l) sequence whose 
covariance matrices are governed by relations 

J J 

P0c+l,k.+l|k) -l £ $ . (k.+l , k) P (k-i , k-j | k)4> ! (k+l,k) 

1=0 j =0 3 

+ r(k+l,k)Q(k)r' (k+l.k) (2.61) 

Proof ; From Theorem 2.1 and Eq. (2.16) 

x(jk+l|k) = E {x(k+l) | z(l) , ... , z(k)} (2.62) 

J 

x(k+l) = E <{> . (k+1 , k) x (k~i) + r(k+l,k)w(k) + ^(k+l,k)u(k) 
i-0 

The substitution of Eq. (2.16) into Eq. (2.62) yields 
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( 2 . 60 ) 


( 2 . 16 ) 


J 

Jt(k+l|k) « E{ t <K(k+l,k)x(k-i) + r (k+l,k)w(k) 
i«0 X 

+ i/>(k+l,k)u(k)| z(l) , ... , z(k)} 

» * 

$ 

t 

J 

“ 2; <J>.(k4d,k)E{x(k-i)|z(l), ... , z(k)} 

t“0 1 

i* a 

+ rCk-+l^)E^Ck)|za), ... , Z(k)> 

+ ij,(k+l,k)fc{u(k)|z(l), ... , z(k)} 


(2.63) 


The application of; property. ,(2. 41) and Eq. (2.7) to the second term 
causes It to vanish. 

E fw(k) | z(l) , ... , z(k)} « E{w(k)} - 0 (2.64) 

» * 

Under the assumption that the control sequence {u(k) , k=0, 1, ... } is 

known or can. be specified as desired, the third term in Eq. (2.63) be- 

* 

comes 

* ♦ 

4> (k.+l , k.) E £u (k) | z (1) , ... , z 00 }■ * «(k.+l,k)u(k) £2.65) 

* 

The substitution of Eq. (2.64) and Eq. (2.60) reduces Eq. (2.63) to 

* / 

J 

*(k+l|k) = 2 ; <$> . (k+1 , k) £ (k-i | k) + tf>(k+ l,k)u(k) (2.66) 

i=0 

which verifies Eq. (2.60) of Theorem. 2.2. from, the definition of pre- 
diction error (2.52), filtering error (2.53) and smoothing error (2.54), 
application of Eq. (2.V16) and Eq. (2.66) yields 


♦ 
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x (k+1 | k) * x (k+1 ) - £ (k+1 | k) 


t <t> j (k+1 , k) x (k-i) + r(k+l,k)w(k) + ip (k+l,k)u(k) 
i^O 


w 

— £ <j > * (k+1 , k) & (k— i j k) - ij> (k+l,k)u(k) 
i«0 . 


w 

,E 4>.(k+l,k)x.(k-t|k) + r(k+l,k)w<k) 
1*0 1 ' 


(2.67) 


It remains to establish, that x(k+l|k) Is a zero mean Gauss-Markov 

» • * 

•^(j+1) sequence. This can be done by examining Eq„ (2.67). Since w(k) 

is gaussian and Ix(-i|o), i ■ 0, 1 J> is gaussian, it follows 

that x(k+l|k) is a zero-mean discrete-time, gaussian sequence. The 
Markov property also follows immediately from Eq. (2.67). In fact this 
has a Markoy -(J+1) property since the sequence depends on events occur- 
ring (J+l) time intervals in the past. 

*. * * 

Now the expression may be determined for the prediction error 
covariance matrix. 

P (k+1, k+1 1 k) = E(x(k+l|k)x , (k+l|k)} 


E{[£ (J) (k+l,k)x(k-ifk) + f(k+l,k)w(k)] 

i te 0 * 


[ t <{> (k+i,k)x(k-j|k) + r(k+i,k)w(k)] 1 } 

j-0 J 


J J 


l l <J> i (k+l,k)E{x(k-i| k)x‘ (k-j| k)}(^ f . (k+l,k) 
i=0 j=0 3 

J 

+ 2 £ 6 . (k+1 , k) E { x (k- i | k)w' (k)}T 1 (k+l,k) 
i-0 * 


+ r (k+i , k) e(w (k)w' (k)) r ■ (k+i , k)} 


( 2 . 68 ) 


From the definition of filtering error, Eq. (2.53), and smoothing 
error, Eq. (2.54), 

E{xCk.-i|k)v? t (k) } » E{[x(k~i) - x(k~i|k)J [w* (k) ] } •' 

« E{x(k~i)\?' (k) ) “ E &(k~i|k)w* (k) > (2.69) 

The first term of Eq. (2.69) is identically- zero due to property (2.40). 
Further, since x(k-i|k) is a linear combination of the measurements it 

m 

may be expressed as 

k 

x(k~i|k) * i A(i)z(*) (2.70) 

i**i * . 

■ ► 

The substitution of Eq. (2.70) in the second term of Eq. (2.69) causes 
it to vanish, also* 

* 

. k 

E{x(k-i|k)w f (k)} * E( % A(i)z(l)w t (k) ) 

• , ' ^ 1 . 

k 

* E A(i)E (z (i)w 1 (k) } 

A~1 

« 0 (2.71) 

The application of property (2.41) causes the cross-product terms of 
Eq. (.2.68) to vanish. Thus Eq. (2.68) becomes 

J J 

P (k+1 ,k+l | k) =• E E 4> . (k+l,k)E{x(k-'i|k)x' (k-j |k) }<{>‘ (k+1 |k) 

• i=0 j=0 X ■ 

+ r(k+l,k)E {w(k)w' (k) }T * (k+1 ,k) 


The substitution of Eq. (2,44) and Eq. (2.8) gives 

J J 

P(k+l,k+l|k) * £ 1 <{, .(k+l,k)ECk~l,k^|lc)(|»’(k41,k) 

i*0 j«0 1 J 

* 

+ r (k+l.lOQOOr (fc+l.k) (2.72) 

'* 

Note, ?,at this point, that the solution to the single-stage opti- 

* * * 

mal prediction problem is solved, The associated error covariance matrix 

* • | 

expressed by Eq. (2.72) requires knowledge of P(k-i,k-j | k) , however, and 

• 

this is known only for k « 0. In the next section this problem is re- 
' solved and expressions for the filtering -and smoothing error covariance 

matrices P(k-'l,k-^| k) are obtained. This information is then combined 

* ». 

with that of the single-stage optimal predictor to obtain the optimal 
filter for time-delay systems. 

2.6 . Optimal Filtering for Discrete Linear Systems With Time Delay 

In developing the algorithm for optimal filtering for the sys- 
tem of Eq. (2.37) and Eq. (2.38) it is assumed that only the initial 

estimate {k(-i|0) - 0, i-0,1 J} , the covariance matrices of the 

filtering and smoothing errors at the initial time P(-i,-j|0) - 
E{x(-i| 0)x‘ (-j 1 0) ) - E{x(-i)x(-j)> = P(-i,~j) and the set of measure- 
ments' (z (1) , ... , z(k), z(k-fl) , k > 0} are given. 

From Theorem ’2.1 the optimal filtered (i-0) and smoothed 
(i— 1 , • . . | J ) estimates k(k-M-i|k+l) are given by the relation 

£(.k+l~i|k+l) - E {x (k+l-i) J z (1) z(k), z(k+l)} i=0,l,...,J 
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* 
t 

i 

* * 

• i 

Property (2.26) may now, be applied to examine (2.73). 

♦ 

mr 

fc{xj)r,z} « £{x| y) + fe{x| z) - x “ (2.26) 

* 

. / 

where z z - E{z| yl . 

By substituting (2.26) into (2.73) and noting that x « (k+1 ,k)u (k) 

% 

JfcCM-l-il k+1) = E{x(k+l-i)| z(l), . .. , z(k)> 

+ E{x(k+l-i)| z(k.+i|k)} - \\> (k+l,k)u(k) (2.74) 

for k 0, 1, ... and where 

■f * ■ 

* 2 (k+1 1 k) « z (k+1) - z(k+l|k) 

** zCk+1) - E{z(k+l)| z(l)., , z(k)) (2.76) 

This difference z(k+ljk) is called the measurement residual . By sub- 
stituting Eq. (2.38) and solving for z(k+l|k) 

z (k+1 1 k) = E (H (k+1 ) x (k+1 ) + v(k+l)| z(l) , ... , z(k)> 

*= H.(k+1 ) E {x (k+1 ) j z (.1 ) , ...» z (k) } 

+ E{v(k+l)| z(l) , ... , z(k)) 

« R(k+l)x.(k+l|k) + E(v(k+1) | z (1) , ... , z(k)) (2.77) 

The second term vanishes by Eq. (2.43) and Eq. (2. 77) becomes 

2 (k+1 1 k) = H(k+l)x(k+i| k) > k r 0, 1, ... C2.78) 

Uith these preliminaries completed, the basic theorem for optimal filter 
ing of disdf^ce linear systems with time delay may now be proved. 
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Theorem 2.3 . (a) The optimal filtered estimate 

5c (k+1 1 k+1) for the system described £>y Eqs. (2,37) and 
(2.38) is given by the recursive relation 

Jr 

x (k+1 | k+1 ) = x(k+ljk) + Kg(k+1) [z(k+l) - x(k+lj k)') (2.79) 

which is a specific case of the more general expression 
*(k+l-i|k+l) = x(k+l-i|k) + K i 0c.+l)[z(krM) - x(k+l|k)] (2.80) 

for k = 0, 1, ... where x(-i| 0) = 0, i = 0, 1, . . . , J. 

(b) K^(k+1) is an n x ra matrix which is speci- 
fied by the set of relations 

K^Oc+1) « P (k+l-i , k+1 | k) H. 1 (k+1 ) I H,(k+1 ) P (k+1 , k+1 | k) H ’ ( k+1 ) 

+ ROt+Dl"* 1 i = 0, 1 J (2.81) 

J 

P(k-j,k+l|k) = £ P (k-i , k-j | k) 4» ! (k+1 , k) j = 0,1,..., J-l (2.82) 

1=0 x 

P(k-i,k-;j|k) « P (k-i , k-j | k-1 ) - K^(k)J(k)P(k,k-j |k-l) (2.83) 

i » j “ 0 , 1 , ... , J 

J J 

P (k+1, k+1 Ik) = z z <j> . (k+1, k)P (k-i, k-j I k) (j, * (k+1 ,k) 

1=0 j=0 x 

+ r(k+i,k)Q(k)r t Ck+i,k) (2.84) 

for k = 0, 1, ... and P(-i,-j |0) = P(-i,-j) is the ini- 
tial condition for Eq. (2.84)= 

•— * .■ * 

(c) The stochastic process 

{x (k+1 1 k+1) , k = 0, 1, ...) which. is defined by 
x (k+1 1 k+1) == x (k+1) - x (k+1 1 k+1), k = 0, 1, ... 
is a zero mean Gauss-l-Iarkov -(J+l) sequence whose co- 
variance matrix is. given by Eq. (2.83) for i = j = 0. 


' . ..L,..: ,Ui-; :. 


Proof : Prom Eq. (2.74) 

x(k+l-i|k+l) - x(k+l-i|k) + E {x (k+l-i) ( z (k+1 1 k) } - i|»(k+l,k)u(k) 

(2.85) 

» * t * 

Further, since x(k+l) and z(k+l|k) are gaussian, property’ (2.35) may be 


invoked 


Elx|z} « x + P ~P~~z 
1 xz zz 


(2.35) 


to obtain the following result 


*(k+l-i|k+l) « if>(k+l,k)u(k) + £ (k+l-i, jk) + E & (k+l-i) z 1 (k+1 1 k) } 

• [E {z (k+1 | k) z ' (k+1 1 k) } J*z (k+1 1 k) - ip (k+1 ,k) u (k) 

( 2 . 86 ) 


By defining K^(k+1) := E{x (k+l-i) z 1 (k+1 1 k) ) [ E (z (k+l| k) z ’ (k+1 1 k) ) ] 

i - 0, 1, . . . , J 
Eq. (2.86) may be rewritten 

x(k+l-i|k+l) = x (k+l-i | k) + K 1 (k+l)z(k+l|k) 

However, it is clear from Eq. (2.75) and Eq. (2.78) that 4 
z (k+1 1 k) = z (k+1) - z (k+1 1 k) 

*=■ z(k+l) - R(k+l)x(k+llk) 

The substitution of Eq. (2.89) into Eq. (2.88) yields 


(2.87) 


( 2 . 88 ) 


(2.89) 


£ (k+l-i | k+1) = & (k+l-i | k) + K. (k+1) [z (k+1) - H.(k+l)£(k+l|k) 3 

(2.90) 

which is the result postulated in .Eq. (2.75). The appropriate initial 
conditions are obviously {k(-i(0) = 0, i = 0, 1, ... , J). This result, 
Eq. (2.90), combined with that of th.e one-step predictor, Eq. (2.66), is 
sufficient to describe the structure of the filter which is shown in 
Figure 2.2. 


= t <f> . (k+1 , k) £ (k-i | k) + (k+1 ,k)u(k) 

. i-0 


( 2 . 66 ) 








Evaluation of K. (k+1) From the definition of prediction error and Eq. 
(2.8 9.) 

%(k+l|k) * z(k+l) 7 &(k+l|k) 

„ # 

* / 

« z(k+l) - H.(k+1 ) x (k+1 | k) 

» •% * i 

« H(k+l)x(k+l) + vCk+1) - H.(k+l)k(k+l| k) 

* ? « H(k+l)x(k+l|k) + v(k+l) (2.91) 

Consequently, 

*i(W-l)«<fcH) ■= E{z(k+l|kr Z >(k+ 1 |k)} , 

i E I I R(k+1 ) x (k+1 [ lc) + vCk-+l)][x , Ck+l|k)H’(k+l) 

+V , (k+1)]} (2.92) 

« H(k+l)E{xOc+l|k)x l (krKL j k) }E’ (k+1) 

' ’ , + H.(k+1 ) E {x (k+1 1 k) v 1 (k+1 ) } 

+ Efv-Ck+Dx- 1 (k+1 1 k) >H* (k+1) 

+ E {v(k+l) v ’ (k+1) > (2 . 93) 

The middle two terms of Eq. (2.93) are now shown to vanish.. Since one 
is just the transpose of the other it is sufficient to examine only one. 

E ix (k+1 J k) v x (k+1 ) } = E (x (k+1) v’ (k+1) > - E{x(k+l|k)v’ (k+1) } (2.94) 
From Eq. (2.43) the first term of Eq. (2.94) vanishes. From Eq. (2.23) 
and Eq. (2.27) the estimate is a linear combination of the measurements 

k 

k Ck-i+1 1 k) = 2, A.(i)z(£) i^.O, 1, ... , J (2.95) 

• £,-1 

The substitution of Eq. (2.95) in the second term of Eq. (2.94) yields 

k 

9 E ft (k+1 1 k) v 1 (k+1 ) } = E { 2 A o a)z(t)v' (k+1) } 

tf* 1 
k 

• 2 A 0 OOE{z(j>)v f (k+1)} « 0 

i=l 


(2.95) 
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by the property expressed in Eq. (2.43). Therefore, Eq. (2.93) may be 
rewritten, using Eq. (2.10), 

' P z( k+ I)z( k+ 1) - H(.k+l)p(k+l,k+l| k)lL' (k+1) + R(k+1). ' (2.97) 

Evaluation of P ~ in Eq. (2.35) and substitution of Eq. (2.89) yields 
P x2 ■= fe^k+l-i^^k+llk)) • 

~ EOx(k+l-i| k) + x(k+l-ij k) ] [x 1 (k+l| k)R f (k+1) + v'(k+l)j} 

« E(x(k+l“i| k)x' (k+l| k)}R' (k+1) 

* . + E{x(k+l-i|k)v’ (k+1)} 

+ E lx (k+l-'i| k) x * (k+1 | k) } R 1 (k+1 ) 

+ E{*(k+l~i| k)v' (k+Dl (2.98) 

The second term in Eq. (z. 98) vanishes due to Eqs. (2.42), (2.96) and 

(2.58). By property (2.33) and Eq. (2.70) the third term vanishes. The 

* * 

fourth term in Eq. (2.98) vanishes due to Eq. (2.96). Therefore, Eq. 

(2.98) becomes . , . 

^x (k+l-i) z (k+1 1 k) “ ® ^ (k+l~i| k) ^ Ck+1 1 k) } R (k+1 ) 

« P {k+l-i , k+1 1 k} R l (k+1 ) (2.99) 

The substitution of Eq. (2.97) and Eq. (2.99) into the defining relation, 
Eq. (2.87), for K^(k+1) gives 

K^k+l) * p x (i c+ i-i)z(k+i|R) P z(k+l)z(k+l) 

= P (k+l-i , k+1 1 k) H 1 (k+1 ) [ R (k+1 ) P (k+1 , k+1 1 k) H f (k+1 ) 

+ R(k+1)3~ 1 (2.100) 

for k = 0, 1, ... and Eq. (2.109) of Theorem 2.3 is proved. Note that 
if R(k+1) is assumed to be positive definite, it follows that the re- 
quired inverse always exists since P(k+l,k+l |k) is positive semi- 

% 

a 

definite. 


33 


Evaluation, of P(k+l-i,k+ll k) 

From Eq. (2,72) the expression for P(k+l,k+l| k) is known to be 

J J 

P(.k+l,k+l|k) n % z <p , (k+1 , k) J? (k-i , k- j | k) A ! (k+1 , k) 

i*=0 j«0 x 3 

+ rCk+l.^QCkJr'Ck+l.k) (2.72) 

It remains to determine the error covariance matrices 

V (k+l-i, k+1 1 k) and P(k-i,k-j |k) . For i - 0, P(k+l,k+l|k) is defined by 

» 

Eq. (2.72) so the cases of interest are for i = 1, ... > J. Now to de- 

i 

termine F(k+l-i,k+l |k) for i = 1, ... , J. From Eq. (2.37) and Eq. , 
C2.66) 

X(k+l|k) * x(k+l) « x Ck+1 1 k) 

J 

* 1 $ i (k+1 , k) x (k~i) + r (k+l,k)w(k) + ^(k+l,k)u(k) 

1=0 

il- J 

- X <J> i Ck+l,k)kCk-i|k) - ip (k+1 , k) u (k) 

1=0 * 

J ■ 

£ $ j (k+1 , k) x (k-i j'k) + r(k+l,k)w(k) (2.101) 

1=0 . 

From the definition of error covariance matrix and Eq. (2.101) 

P (k-j , k+1 1 k) = E (x (k- j j k) x 1 (k+1 1 k) 

[j=0,...,J-l] 

J 

= E lx (k-j | k) [ w ’ (k) r ’ (k+1 , k) + l x ' (k-i| k)4> ’ (k+1 , k) 3 } 

i=0 

= E(x(k-j | k)w f (k)}r * (k+l,k) 

J 

+ l E{x(k-j| k)x* (k-i| k)}(J> ! (k+l,k) (2.102) 

1=0 x 

The first term in Eq. (2.102) vanishes due to Eq.. (2.40), Eq. (2.95) and 
Eq. (2.41). Therefore, Eq. (2.102) becomes 
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J 

P(k-J,k+l|k) « t P(k-j,k~i|k)<j>’(k+l,k) j * 0, 

i K 0 * 

wliich. completes the proof of Eq. (.2.84) of Theorem 2.3. 

Recall that for i c 0, 

P(k+l~i,k+l|k) «= P (k+1 ,k*KL | k) 

which is already known from Eq. (2.72). 

* 

* * » * 

Evaluation of Pfk-d^k-j Ik) 

.Finally, it remains to determine, the error covariance matrix 

* 

P(k-i,k-j | k) *» E {x(k-i|k)x 1 (k-j | k) } (2.104) 

*■ * t , 

From the definition of estimation error and substitution of Eq. (2.90) 
and Eq. (2.38) 

x (k-i | k) » x(k-i) ~ x(k-i|k) 

« x (k-i) - [S(k-i|kra) + K 1 (k){z(k) - H(k)*(k|k-l‘) }] 

. ** x(k-i|k-l) - K^(k) [E(k)x(k) + v(k) - H(k)&(kjk-1) 3 
« x(k-i|k-l) ^ K^(k)E(k) x (k| k-1) - Ki(k)v(k) (2.105) 
Substitution of Eq. (2.105) into (2.104) yields 
P(k-t,k-j|k.) = E{XCk.-'i|k-l)S ! (k-.j|k.-l) 

a ! 

- E {x (k-i | k-1 ) v 1 (k) } K \ (k) 

- K i (k)E(k)E£5t(k|k-l)X(k-j|k-l)} 

. -f K t (k)H(k)E£x0c|k-l)x' (k| k-l)}H' (k)Kj (k) 

• + K i (k)H(k)E{xCk|k-l)v , (k)}kj(k) 

- K^(k) E Iv (k) x ' (k-j | k) } 

+ K . (k) E { v (k) x (k| k-1) } Il r (k) K (k) 

' ■ .. ' J 

• . + K^WEMkW'OOJK (k) (2.106) 



X i • # • | J~1 

(2.103) 
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ha th i*d, sixth, seventh and eif , Kt;Ji 
Vanishes , Tfms Hq> (2 ’ mS ******** Eq. < 2 . S4) 

P/s., 7 be bitten 

-i.M/h) = P(k - U . j|H) 

* ' ' 

» 

-P&-i.k/k~ Dlt-toKja) 

" ^ H W P (k, k-j I k-1) 

; i (k)H(k)p(k,k|fc.-i) H t ( fc j K , ^ 

But from Eo /, +K l^R(k)K'(k) J 

Wm E q. ( 2 . 100 J J ( 2 ,lo 

“ "" ,h * * Rft,ri ■ 

terraS and *• C2.W7) becomes . . CanCElS “* «* fifth 

PCk-i.k-j |t)«i(kq t , 

**ich completes th ' ' ^ I k -n , 

the Pr °° f C2 . 83) o ; _ J,k l) «- 108 ) 

. Finally lf „ , * Tfle °rem 2.3. 

- a sevo mean ^ ' ^k + l , k+1> k Q 

mean Ga Mss~Harkov -(jin * ' I/#fc « 0, 1, ... } 

Eq, (? mss ® &< l u ^n.ce, SubstituM ,s 

H W.105) yields Wti.on of Eq. (2.67) 

*^ C+i “'i|k+l) t- . / 

‘ 1, ) ~ K i^)H(k + l) i(k+l[k) • 

iv i Ck+l)v({ c -fi) 

(k+l-ljk) - K^k+DHfk+Df I * ( k+1 ‘ . 

i=o i 1,k)x( Mk) 


- P &+l,k)w(k)J - K i( k + l )v(k+1) 

^ A _ 


b F definition »&> and ( . 

*»*«* sequences It ' ^ ^^ent zero m 

Ic-n ItVM sho >™ in- Section , * me ^n Gauss- 

1 - ) i — ea „„ u2 ' 5 ^ * 0 . 

independent of „ (k) and ° V (J+i > ^quence which is 

. “•»*». <a|a> „ . „„ „„ 
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random n-vector independent of w(k) and v(k+l) for all k =• 0, 1, 

In addition, [E{x(.-i| 0)) , i -• 0, 1, ... , Jj is a zero mean random 
n-vector independent of w(k) and v(k*KL) for all k *= 0, 1,/'... by defini- 
tion and Eqs . (2.19) and (2.20). However, x(i|o) is not independent of 

i' ! i' V # ! 

{x(-i|0), i « 0, 1, ... , j) as is shown by (2.67). Therefore, it must 

be concluded that {x(k+l-ij k+1) , i « 0, 1, . . . , j)*is a Gauss-Markov 

» 

-(J+1) sequence. This concludes the proof of Theorem 2.3. 

2.6. 1 Estimation in Systems With. No Tima Delay 

It is of interest to examine the results of Theorem 2.3 for the 

* 

* 

case where there- is no time delay. This is easily done by setting J *= 0 

in Theorem 2.3 and results in the following theorem: 

* 

Theorem 2.4 , (a) The optimal filtered estimate x(k+l|k+l) 

» 

for the -system described by Eq. (2.5) and Eq. (2.6) is 

t 

* 

given by the recursive relation 

xCc+l|k+l) = x (k+1 1 k) + K(krfl) [z (k+1) - x(k+l|k)] (2.109) 

for k *= 0, 1, ... where x(olo) - 0. 

(b) K(k+1) is an n x m matrix which is specified 
by the set of relations „ . 

K(k+1) - PCk-fllk)^ (k+1) { R(k+1 ) P (k+1 1 k)H‘ (k+1) + R(k+1)]"' 1 * 

( 2 . 110 ) 

P (k+1 I k) = (J) (k+1 , k) P (k | k) <£ * (k+1 , k) + r(k+l,k)Q(k)r’ (k+l,k) 

( 2 . 111 ) 

P(k+1 1 k+1) = [I - K (k+1 ) H (k+1 ) ] P (k+1 ( k) (2,112) 

for k = 0, 1, ... , where I is the n x n identity matrix 
and P(0|0) *=• P(0) is the initial condition for Eq. (2.111). 



(c) The stochastic process' {x(kfl| k+1) , k«0,l,,.,} 

which Is defined by the. filtering error relation 

* # 

5c(k+l|k*M) « x(k-fl) - xOoKL|k+l) is a zero mean Gauss- 

/ 

Harkov sequence whose covariance matrix is given by 

* - 

Eq. (.2.112). 

Thus the results of Theorem 2.3 reduce to the familiar Kalman 

• . 

filter when no delay is present in the system. 

* * * 

2.7 . Computational Aspects 

One of the significant features of the filter developed in 
Sections 2.5 and 2.6 is its recursive form* The measurements can be 
processed as they occur and there is no need to store any measurement 

data. In fact, so far as storage of the measurement . and state is con- 

* 

cerned, only I&(k-i|k) , i - 0, 1, ... , J} need be stored in proceeding 

from time k to time k + 1. ‘ 

The information flow in the filter can be discussed by consider 1 

ing the block diagram of Figure 2.2. 

Suppose that {x(k-i|k), i - 0, 1, ... , J) is known for some k 

and that it is desired to determine x(k+l|k+T) given z(k+l). The compu 

% 

tational cycle would proceed as follows: 

1. The estimates {£(k-i| k) , i = 0, 1, ... , J) are "propagated 
forward" by preraultiplying them by the state transition matrices, 
<J>^(k+!,k), The sum of these is added to (k+l,k)u‘(k) , giving the pre- 
dicted estimate k(k+l{k). 
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2. & (k+l | k) is premultiplied by U(k+1) giving z (k+l | k) which 

* * * * 

is subtracted from the actual measurement z(k+l) to obtain the measure- 

■ ■■ ■■ H IM U IIIM 1 11! H* 

ment residual z (k+l Ik). 

— — — - - ■. , i. / 

i 

3. The residual is premultiplied by the matrix Kg(k+1) and the 
result is added to x(k+l|k) to give x(k+l|k+l). At the same time, the 
delayed residual z(klk-l) is premultiplied by K^(k) . The ith sum is 
added to various delayed sums as indicated in Fig. 2.2 to give 

‘ [x(k+l~i|k+l), i = 0, 1, ... » J}. 

4. fx(k+l-*i| k+l) , i ** 0, 1, ... , J} is stored until the next 
measurement is made at which time the cycle is. repeated. 

The interplay between prediction, filtering and smoothing is evi- 
dent at this point. It can be observed that each estimate is obtained 
using the other. The filter equations a, re 

J 

*(k+l|k)’ = Z $ , (k+l , k)x (k-di k) + iKk+l,k)u(k) (2.113) 

i«0 x 

x(k+l-i| k+l) * 5c(k+l~l|k) + ^.(k+l) [ z (k+l) -x (k+l| k) ] (2.114) 

To initiate filtering, {5c(-i| 0) ^ 0, i ^ 0, 1, ... , J) is used 

4 * 

and Eq. (2.114) can be solved. The equations then proceed recursively 
as described in the four steps above. 

Consider next the computation of the filter gain matrices 
fK.^(k+l) , i - 0, 1, ... , J) and the three covariance matrices 
P (k+l, k+l | k) , {p(k-j ,k+lf k) , j = 0, 1, ... , J-l) and {P(k-i,k-j| k) , 
i,j =0, 1, ... , J) . The relevant equations are • 
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J J 

P (k+1 , k+l| k) * I H ♦ 4 (k+X,k) P <k-i, k-j | k)* J (k+1 , k) 

i*0 j«0 J 

+ r(fcH*k)QCk)r f Cl6H l k) (2.115) 

’ 

* # 

K.^Ck+1) « P(k+l~i l k+l|k)H l Ck+l)[ECk+l)PCk+l,k+l|k)H , (k+l) 

+ RCk+1)]’' 1 , i * 0, 1, ... , J (2.116) 


J 

P(k->j,k+l|k) « l P (k-j , k-i| k)4* ! (k+1 , k) , j « 0, 1, ... , J-l 

lR ° (2.117) 

P(k-i,k-j|k) « P(k"-i t k-j | k-1) - _ (k) H (k) P (k , k-j | k-1 ) 


1, j « 0, 1, ... , J (2.118) 

for k = 0, 1, ... , with. PC-1, -j 1 0) « E frC-sOx* C-j) }; i, j-0,1, . . . ,J. 

r * 

. A typical computation, cycle would proceed as follows : 

* * 

1. Given P(k|k), Q(k), 0^(k+l,k), i - 0, 1 J} and 

P(k+l,k); P(k+l,k+l|k) and' {P(k-j ,k+l|k)', ' j = 0, 1, ... , J-l) are com- 
puted using Eq. C2.115) and Eq. ,(2*116) respectively. 

2. P (k+1 , k+1 1 k) , {p (k- j , k+1 1 k) , k = 0, 1, ... , J-l), H(k+1) 

and R(k+1) are substituted into Eq . (2.117) to obtain (iC, (k+1) , i=0,l,..,j) 

^ ' - J 

which, is used in Step 3 of the filter computations discussed in the 
previous paragraphs. 


3. P(k+l,k+l|k);{P(k-j,k+l|k), j = 0, 1 J-l},' & (k+1, 

i s 0, 1, ... , J) and H(k+1) are substituted into Eq. (2.118) to yield 
(P (k-i+1 ,k-j+l | k+1) , i, j = 0, 1, ... , j). These values are stored 
until the time of the next measurement when the cycle is repeated. 

The matrix inverse which must be computed in Eq. (2.117) gener- 
ally poses no real problem. The matrix to be inverted is m x m, where 
m is the number of elements in the measurement vector* In most systems 



/ 

i 

» 

* » 

# • 

B\ is kept small to avoid the high, cost oi; complex instrumentation.. 
Consequently, it is not unusual to encounter systems with 20 state 
variables but only 2 or 3 measurement variables. 


/ 




CHAPTER III 

OPTIMAL CONTEIOL OF LINEAR DISCRETE-TIME STOCHASTIC, 'SYSTEMS 

WITH. TIME DELAY 

♦ 

* * 

3.1 Introduction 

' i 

The. problem, considered in this chapter is that of controlling a 
system which. is subject to disturbances and measurement errors such that 
some measure of the system’s behavior is optimized. It is shown, in 
addition, that the results may also be applied to the deterministic case. 
Neither, the stochastic nor deterministic results obtained here have ap- 
peared in previous literature. 

» 

Section 3.2 establishes the particular class of problems to be 
solved, the system model being that defined in Chapter 2. The perfor- 
mance measure is the expected value of a quadratic form in the state, 
and control variables over a fixed interval of time. The resulting 
problem is called the stochastic linear regulator problem . 

Section 3.3 reviews previous work in the area of control of 
time delay systems. The majority of the work has been done for open- 
loop deterministic continuous time systems. Very little appears in the 
literature on discrete-time systems and even less on the stochastic 
control problem for systems with time delay. 

Section 3.4 introduces the concept of optimality and applies it 
to obtain the solution to the stochastic control problem. In Section 
.3.5 the computational aspects of implementing the algorithm are presented. 
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% 

( 

t 

» * 

In Section 3,6 those, results are converted tp that for a deterministic 

* 

system and the separation principle of estimation and control is dis- 
cussed in light of these results. The separation principle states that 
the optimal control system consists of the optimal filter in cascade 
with the deterministic optimal controller. The result is also shown to 

reduce to the standard optimal controller when no time delays are pre- 

# 

sent in the system. 

3,2 Problem Formulation 

1 -i - -i |_ _ - . I l». * 

. In this section the system model is presented as in .Chapter 2.* 
The quadratic performance measure of interest is defined. Physically 
realizable controls are discussed and the problem statement formulated. 

3.2.1 System Model 

The system model is defined by the relations 

J 

x(k-KL) = E tp- (k+l,k)x(k-i) + rCk+1 ,k)w(k) • + ^(k+l,k)u(k) (3.1) 

i=0 

* . z(k+l) * H.(k+l)x(k+l) + v(k+l) (3.2) 

for k = 0, 1, 2, ... . The model is the same as that formulated in 
’Section 2.2.3 and for which the optimal estimation problem was solved. 
The following definitions and properties are repeated for reference, 
x ~ n vector (state) 
u - r vector (control) 
w - p vector (plant disturbance) 
z - m vector (measurement) 
v = m vector (measurement disturbance) 


f 
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.* 

6. « n k n (state transition matrix associated with the 

-L, , * 

delayed state vector x(k-i)) 

T «s n x p (disturbance transition matrix) 

... f 

n x r (control transition matrix) 

» 

tt ** m x n (measurement matrix) 

* $c(-i)» i«0,l, , ... ,J} 82 zero mean gaussian random n vector with 
* • / 

positive semi-definite covariance matrix 

Pt-i,-j) * EfcC-Dx'H)}, i,j=0,i j 

(3.3) 

fr(k) , k=0,l,2*. . . } « zero mean gaussian white sequence which. 

is independent of {x(-i) , i**0,l, . . . , J) 
and has a p x p positive seraidef inite 
covariance matrix Q(k) , kfO,l,.«. . 

(3.4) 

' Iv(k+1), k=0 , 1 , . . . } - zero mean gaussian white sequence which 

is independent of* {x(-i) , i-0,1, . . . > J} 
and {w(k), k=0,l, . . . } and has an m x m 
positive semidef inite covariance matrix 
R(k+1) , k=0,l (3.5) 

’ fu(k), k=0 , 1 , . . . } - control* sequence which is either knojm 

or can be specified as desired. (3.6) 
The following properties are recalled from Section 2.5.1 and are needed 
for the development of Section 3.4. 

1. *{x(i), i = 0, 1, ...} is a Gauss-Markov-(J+l) sequence (3.7) 

2. x(i.) and.w(i) are statistically independent for all 

. i = 0.,- 1 , , . . . ' 






-'V 
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3. z(i) and w.(j) are statistically' independent lor all j>i, 

* > 

1 « 1 , 2 , ... , ( 3 . 9 ). 

In addition to these properties, the system equations (3.1) 

* 

and (3.2) must be "controllable," A discussion of "controllability" 
is presented in Appendix A. In general, a system is said to be control- 
lable, if, for the corresponding deterministic system, any initial 
state x(0) can be transferred to any final state x(t^) in a finite time, 

Jl, 0, by some control u. If a system is not controllable, then there 

* 

is no guarantee that a control sequence (u(k), k = 0, 1, ...} can be 
found which transfers the system to some desired final state from arbi- 
trary initial conditions. 

3,2.2 Performance Measure 

A control sequence {u(k), k - 0, 1, ...} is to be constructed 

/ 

to control, the state (x(k), k - 0, 1, . . . } of the system over some 

* 

fixed interval of time [0,N], N - positive integer, such that the per- 
formance measure 

N 

J N - E{ I [x’(i)A(i)x(i) + u f Ci-l)BCi-l)u(i-l)]} (3.10) 

i=l 

is minimized. A(i) and B(i) are, symmetric positive semidefinite matrices' 
which are n x n and r x r respectively and E{*} denotes the expected 
value operation. The expectation is over x and u. 

Jjj is usually interpreted as a "system error plus control effort" 
measure of performance. The first term on the right-hand side of Eq. 
(3.10) implies that the desired state is zer*. If at each point i, the 
desired state is some arbitrary x^Ci). then x(i) would be replaced in 
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r * 

Eq. (3.10) by x(l) - x^i). 

Although, x (i) m 0 £ 01 ; the system model of this paper, this is 

* 

not a . requirement for obtaining a solution. Unfortunately, if x^Ci) ^ 0, 
the mathematical development is quite complicated and obscures the basic 
results. This is obvious from the results obtained by Williams ^ for 

the case where no time delays are present. Thus, as a matter of mathe- 
matical convenience x^Ci) is chosen equal to zero. Note that the quad- 
ratic nature of the term implies that the pleasure of error here is one 

» 

of error-squared and actually of weighted-error-squared because of the 

freedom in choosing A(i) « . 

♦ * 

. The second term of Eq. (3.10) is sometimes called "control energy" 
as a consequence of the quadratic nature of the term. As with the first ' 
term it is referred to as "weighted control effort" because of the arbi- 
trary nature of B(i-1). 

Thus may be viewed as a measure which provides for a trade- 
off between' system error and control input. The relative importance of 
the two terms is reflected in the choice of A(i) and B(i-l). Because 



trol sequence {u(i-l), i - 1, ... , N) which minimizes J^. 
312.3 Physically Realizable Controls 


The control sequence {u(i-l), i - 1, ... , N} which minimizes Eq. 
(3.10) is not arbitrary. For example, the solution may lead to control 

sequences which cannot be mechanized in practice^ such as those which 

* 

require input data that is not physically available when required. Ad- 
ditionally, since it is anticipated that variations in the system's 




state. occur, it is desirable to have. the. control sequence depend upon 

information, which. Is available about the state, namely, the measurements. 
* » 

» 

•Thus the control sequence is to involve feedback. If no plant distur- 
bances were present and if the initial conditions were perfectly known 
then an open-loop control law would be satisfactory, assuming, however, 
the plant is precisely known. Plant and measurement disturbances, how- 
ever, in the system described by Eqs. (3.1) and (3.2) make a feedback con 

* * 

trol law necessary! In the sequel, the. control sequences depend only 
upon information about the system's state which, is available for pro- 


cessing. 

• For any given k « 0 , 1 , ... , N-l it is obvious that the avail- 
able data on the system's state consists of the sequence of measurements 
£z(l), ... , z(.k)} and the mean value of the delayed initial states 
{x(-i) i i = 0, ... , J) . The control vector at k can then be written in 
the form * 


u(k) = yjjz*(k), x*(0)3 
where z*(k) is the mk vector 
z*(k) « z(l)l 


z(k)J 

and x*(0) is the n(J+l) 


x*(0) = x(0) 
x(.-l) 


*(-J) 


(3.11) 


(3.12) 


(3.13) 


vector 
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ancj, Is an r-dimensional vector-valued function of the indicated vari- 

k 

ables. Note that y, is to be determined such, that J vt is minimized and 

. k N 

it is- not necessarily restricted- to be of the same form for all k. 

- / 

A control vector of the type defined by Eq. (3.11) which depends 

only on available data is a physically realizable control and y^» • 

* 

k « 0 , 1 , ... , N-l is a physically realizable control law . For k *= 0 , 
utO) can only be a function of x*(0) since no measurements are available. 
If independent of z*(,k) for all k, then yCk) is an open-loop con- 
trol law . , , 

* *►#»». ♦ 

3.2.4 Problem Statement 

The problem, can now be stated as follows. 

"Determine a physically realizable control law of the form 
(3.11) for the system described by Eqs. (3,1) and (3,2) 

which, minimizes the, quadratic performance measure (3.10)." 

i * 

Such a control lav; is called an optimal control and the problem 
itself is called the discrete stochastic linear regulator problem (with 
delay). The word "regulator" arises because x,Ci) - constant. 

3.2.5 Discussion , 

Three important restrictions on the class of problems are: 

1. The performance index is time independent. The terminal 
time itself may not be part of the performance index. 

2. No amplitude bounds are placed on the control vector. How- 
ever, the second’ term in tends to' limit excessive control. 
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3. The state at the terminal time is not constrained. As with 

* 

the control vector the state at the terminal time may only be affected 

I * 

.indirectly through judicious selection of A(i) in the performance meas- 

* * 

ure. 


3.3. Review of Previous Work on Optimal Control of Linear Systems with 

Time Delay . *• % , 

* 

» 

Attention is restricted below to work, appearing in the litera- 
ture which may be applied to the stochastic control problem with time 
delays in the plant. 

Thus, the large number of papers which develop open-loop control 

* 

are omitted. Similarly, those papers which employ unity feedback and 
vary only the gain in the forward path are not discussed. Briefly, then, 
the papers discussed below have the following characteristics: 

1. Use of state-variable feedback 

2. Presence of delays in plant 

The reader interested in other characteristics of the control of time 
delay systems is referred to the extensive bibliography compiled in 
the bibliography. 

A number of approximate techniques appear in the literature 

(Bibliography: 73, 116, 118, 157, 159, 161, 164, 212). These methods 

7 _q T 

use either a Pade approximation to the time delay which expresses e 

as a ratio of polynomials in s or they express the delayed state x(t-h) 

as a Taylor Series. 

xCt-h) = x(t) - hx(t) + h 2 a(t^ ** 

2 ! 


* i • 
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Once either of these approximations is made, the problem may then be 
treated as one with no delays and the well-known optimal control results 
may be applied. Since, by definition, these methods are approximate, 
they yield a sub-optimal solution and are not discussed further. It is 
recognized, however, that such methods may be very good, depending on 
the accuracy of the approximation. 

♦ , 

In 1969, Eller et al developed an exact deterministic control 

law for continuous systems with time delay. The solution, however, is 

*• 

similar in form to the estimation solution by Kwakernaak^ mentioned ear- 
lier and suffers from the same computational disadvantages. * Both results 

are limited to plants with a single delay, although the authors suggest . 

* 

that the theoretical results may be easily extended to include multiple 
delays. It should be remarked, however, that even for a single delay 
the computation is so excessive that with a large computer (CDC 6600) 

i 

only scalar examples are worked. 

» • * 

g 

In 1969 Koivo derived the solution to the stochastic control 
problem for continuous systems with time delay. He showed it to be the 
same as the result obtained by Eller et al, thus verifying that the 
•separation principle also holds for continuous systems with time delay. 

Larson and Wells ^ overcame" some of the computational problems 
in a paper published in 1969. Attention is restricted to serial systems, 

t 

where the delay is in the forward path only, but this represents a large 
number of practical problems. Their results obtained are optimal only 
for single input-single output systems, however. 




The expanded state representation for discrete systems with 
time delay presented in Chapter 4 of this dissertation was first intro- 
duced by Koepcke 10 in 1964. An alternate form of the expanded state 
representation was later used by Day 11 in 1968. The results achieved 
by both authors require extensive computation and storage. 

The results obtained in the remainder of this chapter can also 

be obtained by examining the submatrices of the solutions of Koepcke and 

* * 

Day. Solutions of these submatrices requires considerably less computa- 
tion and storage time, 

3.4 Stochastic Control Problem 

3.4.1 Problem Formulation 

* 

From Eqs, (3.1) , (3.10) and (3.11) the problem becomes 
J 

x(k+l) ~ 2 tjr. (k+lj.k)at(k-i) + r(k+l,k)w(k) + i{/(k+l,k)u(k) (3.l4) 

z(k+l) ** H(k+l)x(k+l) + v(k*li) 

N 

J N = E{ 2 x'(i)A(i)x(i) + u 1 (i-l)B(i-l)u(i-l) (3.15) • 

i«l 

u(k) = n k U*(k),x*(0)] (3.16) 

i 

where z*(k) is the ink. vector 

(3.17) 
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and x*(0) is the (J+l) vector 

X*(0) = x(0) 

* • 

* 

x(-J) 

mJ 


(3.18) 


In the deterministic case, the expected value would be removed 
from the expression for J^, and the measurement process would become 

z (k+1) = H (k+1) x(k+l). Assuming the system is observable. the state var- 

T . 

iables could be calculated exactly, and the uncertainty associated with 

i 

the delayed initial states would be removed. The estimates 
{£0-i)» i=0, 1, ... , J) would be replaced by the actual values of the 
delayed initial states, 6c(-i) , i=0, 1, ... , J}. The stochastic regu- 
lator problem may now be stated. 

. ,\V .... •»£;./ ' 

--Jt . 

'i\ ’ 

3.4.S Problem Statement 

Determine a control law of the form (3.16) for the system of 
Eq. (3«14) which, minimizes' the performance measure in Eq. (3.15). 

The resulting system has the block diagram which is given in 

* 

Pig. 3.1. The problem is to specify the controller which will operate 
upon the output states, z (k+1) , to determine the control vector which 
minimizes the performance measure. In general, the resulting control 
law could involve feedback of all the preceding values of the measure- 
ment vector. Prom a computational point of view this poses a similar 
problem to that of estimation discussed in Section 2.5, By applying the 
Principle of Optimality to' this problem (discussed in the next paragraph) 
a set of recurrence equations are found which resolve this problem quite 
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handily. Another problem is that of determining the optimal controller 

when not all of the states are available for measurement. The separation 

1 2 

principle, which was first suggested by Kalman and Koepcke- and later 

13 14 

proved by Joseph and Gunchel provides a neat solution to this qtjas- 
tion. This important result reduces the optimization problem to two 
separate optimization problems, one of estimation, the other of control. 
It states that the optimal controller is the same as the deterministic 
controller which operates on the optimal state estimates as' if they were 

i 

the actual values of the states. In this chapter the separation princi- 
ple is modified slightly and also shown to apply to the delay case. The 
stochastic control solution is shorn to rely upon the availability of 
the optimal estimates of the delayed states for implementation. Subse- 
quent comments are made to show that the deterministic controller and 
the stochastic controller are the same. 

3.4.3 Principle of Optimality 

The principle of optimality may be stated as follows: 

■ •» f i" 

Theorem 3.1 . For any initial state and initial con- 
trol, the- remaining optimal control at any subsequent 
time must constitute an optimal ,one for the remainder 
of the trajectory. 

A simple interpretation of the principle of optimality would be 
as follows. Suppose that, for some discrete-time system whose initial 
state is x(-i), the optimal control {u_(t), t j t j t } minimizes some 

V, V 1 Z 

► .$) ■ 

performance measure J over the interval [t^, t^]. Then the principle of 



optimality states that the control {u°(t), t* ^ t i t^} minimizes the 

same J for the same system over the interval [t f , t 2 ] with the initial 

* 

.state x(t*)“ which resulted from u°(t) acting over the interval [t, , t']. 

/ J" 

t 

A proof of Theorem 3.1 is given in Med itch** (p. 331) and is a 

A 

powerful result for use in the solution of control systems optimization 
problems. In discrete-time problems with no delay, the problem can be 

reduced from one of determining an entire control sequence at once to 

* 

one of determining the control as a function of a state of time k based 
on the results at time k + 1. For the discrete time problem with delay 
the same technique again results in. a set of recursive equations for the 
control. 


3.5 Stochastic Control Problem For Systems ‘With. Tima Delay 


is defined tp.be the minimum value of the performance measure 
J N in (3.15), 



= min . e . 
u(0) 


N 

min E { £ x' (i)A(i)x(i) + u’ (i-l)B(i-l)u(i-l) } 
u(N-l) 1=1* 

(3.19) 


The-problem is one of selecting rN variables, namely u(0),u(l), ... , 

u(N-l) to minimize J . A Lagrangian formulation would require the solu 

N 

tion of rN algebraic equation subject to the constraints expressed by 
the system equation (3.14) . Even for modest problems this approach de- 
mands excessive computation. 

The problem can also be viewed as an N-stage decision process 
where the N decisions, u(0),u(l), ... , u(N-l) minimize the quadratic 
cost. By applying the principle of optimality the decisions are made 
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one* at a time, rather than simultaneously, the Nystage problem is re- 
duced to N one-stage problems. This technique starts with the final 
stage of control and uses induction to proceed backwards in time to an 
arbitrary initial time as shown below. 

3.5.1 Single-stage 

Suppose that the problem is simply that of selecting a control 

* 

which minimizes the performance measure for the last stage of control. 

That is, the problem is a single-stage optimization problem 

V* min Efe'(n)A(N,x(N) + u’(N-l)B(K-l)u(N-l)} (3.20) 

1 uCN-1) 

However, from Eq. (3.14) 

J 

x(N) « £ <f>.(N,N-l)x(N-i) + F (N , N-l ) w (N-l ) + ^(N,N-l)u(N-l) 

1=0 1 (3.21) 

Substituting this result into Eq. (3.20) and dropping most time indices 
for convenience 

J J 

V. = min E {[ £ x 1 (N-i-l)<j> ! + u ’ ip ’ + w*r']A[ £ <J> x(N-j-l) 

1 u(N-l) i=0 1 j=0 3 

+ ^u + Tw] + u f Bu] (3.22) 

By noting that the individual product terms are scalars and that A is 

symmetric, the terms may be combined to yield 

J J J 

V = min E‘ {£ £ x* (N-i-l)$ !A|> .x(N-j-l) + 2u^’A £ $ .x(N-i-l) 

1 u(N-l) i=0 j=0 13 i=0 1 

J 

* + 2w’r'A £ $ .x (N-i-1) + 2u*^ f Arw + u'fijj'M+Bju 

^ i-0 

+ wT 1 ATw} 


(3.23) 



I 

I 

t 
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* 

Where the Indicated expected value is over x, w and u. Making use of 
the matrix identity 

✓ 

trace (ABC) = trace (BCA) = trace (CAB) 

/’ 

and since each term, in Eq. (3.23) is a scalar^ the third and fourth terms 
of Eq. (3.23) vanish 


J J 

Etw^'A Z <{>.x(N-i-l)} = E (tr [F 'A Z <p .x(N-i-l)w’ (N-l) ] } 
i^O i^O x 


J \ 

« trace[T'A Z <{> .E {x(N-i'-l)w* (N-l) }) 
i=0 1 


** 0 by property (3.8). (3.24) 

Similarly, Etu^'ATw} = u'^'ArE {w} » 0 (3.25) 

by properties (3.6) and (3.4). 

From Chapter 2 the property of conditional expectation gives 
E{x} = E{E(x|y)}, where the outer expectation on the right-hand side is 
oyer y, thus allowing Eq. (3.23) to be rewritten 


J J 

V *= min E{E[ Z Z x' (N-i-l)4> ' A<j>.x(N-j-l) 
1 u(N-l) i=0 j=0 1 J 


J 

+ 2u’^'A Z <j>.x(N-i-l) + u ? [i/j'A^ + B]u 
i=0 x 


+ 


w* r ' afw 


z* (N-l) ,x*(0) ] > 


(3.26) 


This equation can be minimized by minimizing the inner expected value 
in Eq. (3.26) with respect to u(N-l) for all z*(N-l) and x*(0). The 
physical realizability condition requires that u(N-l) be some determin- 
istic function of z*(N-l) and x*(0). Thus the second and third terras of 
Eq. (3.26) become respectively 




J . 

■E{2uVA l <f> x(N-i-l) z*(n-1),£*(0)} % 

1=0 1 • 

’ J . 

= 2u f ^'A £ <p E{x(N-i-l) z*(N~J ) ,x*(0) } (3.27) 
1=0 1 

and E{u* [ijj'Aijj + B]u| z*(N-l) ,x*(0)} = u * [i/j * Ai^/ + B]u (3.28) 

Now, setting the gradient of the inner expected value of Eq. (3.26) equal 
to zero, 

J 

V x = 0 = 2f*A E 4> ± E (x(N-i-l) z*(N-l) ,x*(0) ) 
du(N-l) i=0 • 

+ at^’Aip + B]u(N-l) ' (3.29) 

and solving for u(N-l), expression (3.30) is obtained. 


u(N-l) = -[i|)’At|j + BJ^'A 2 <j> -x (N-i-1 1 N-l ) (3.30) 

1=0 


where ij> = ^(N,N-1) 

<t> ± - +i<N # N-i) 

B = B(N-l) 

A = A(N) (3.31) 

A 

Notice that the separation principle is evident in this one- 
stage case. The optimal control is a set of gain matrices each of 
which is associated with a. separate filtered estimate, where the gain 
matrices and filters are computed independently. Define 

W Q0 (N) =A(N) and (3.32) 

note that Wqq(N) = Wqq(N) since A(N) is symmetric. 

S i (N-l) = -{•]> 1 A f-:+. 



(3.33) 
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J 

and write u(N-l) « Z S. (N-l)*(N-i-l|N-l) (3.34) 

i«=0 1 4 * 

may now be evaluated by substituting Eq. (3.31) into Eq. (3.23) (less 

the third and fourth terms) . 

* 

J J 

V « E{ Z Z x , *CN-i-l)^!W no *.x(N-j-l) 

1 i=0 j=0 1 

J J . 

-2E E ^ , (N-i-l|N-l)**W 0 ^[^ , W nB]“>'W nri <{» x(N-j-l) 

i=0 j=0 00 j 


+ S « ' Jitf’W tp+B ] -1 [^’Wqq^+b] 

1=0 j=0 1 uu 00 

• [^'W 00 nB] _ %'W 00 4. j *(N-j-l|N-l) + w'r'W 00 rw} (3.35) 

* ■ ■ 

This expression may be simplified by noting that 

* 

*(N-i-l|N-l) = x(N-i-l) - x(N-i-l|N-l) (3.36) 


Then, letting A . denote the n x n matrix 

ij 

, * 

• .\j s n w do^ [ ^ ,w oo^ B J - ^’ w oo* j 


(3.37) 


The second and third terms may be combined in the following way. 

J J 

Z Z [~25c f (N-i-l|N-l) A. .x(N-j-l) + S' (N-i-l|N-l) A. ,*(N-j-l | n- 1) ] 
i«0 j=0 id 

J J 

*= Z £ '[-2(x!-x! ) A .x. + (x!-x!) A- . (x.-x^)] 
i=0 j =0 i 1 - i n i i- ij 3 J 

j j 

“ t Z [**2x! A. .X. + 2x' Ax, + xlA.^x. - x! A..x. 
is0 js=0 i ij J 1 iJ j 1 J 1 ij j 


- x! A. .X. + x! A. .03 
1 ij J i ij 3 


(3.38), 
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The second, fourth and fifth terms cancel since each term is a scalar, 


*= A* . » and the equation results in a double sum where each sum con- 
ij Ji 

tributes the same range of indices. Thus Eq. (3.38) reduces to 

✓ 

J * J , ' 

Z Z - XiA .x ) 

i=0 1=0 J 3 J 


(3.39) 


Therefore Eq. (3.35) may be written 

J J 


V 1 ■ t \i o 


J J 


+ E{ Z E x'Ll} + E{w‘r*W nn 
i=0 j=0 1 3 00 


rw> 


(3.40) 


Since the last two terms of Eq. (3.40) are scalars, these terms become 
J J . 

e{ 2 ; z. x! A + w'r’w^rw) 

1«0 1=0 1 J 00 


J J 

= E{trace[ Z Z x! A .X. +‘w , r'W rt Tw]> 

00 

J J 

* E (trace [ Z Z t A. .X.x’ + r’W nn rww f ]} 
i=0 j=0 3 31 uu 


J J 

- trace[ Z Z s AY.. + T ! W TQ] 

i=o 31 00 


(3.42) 


where P . is the n x n filtering covariance matrix 

ji 

P . ^P(N-j-l,N-i-l|N-l) 3 E{x(N-j-i)x' (N-i-1)} 

w" 

and Q J=Q(N-1) 3 E {w(N-l)w' (N-l) } 

Denote Eq. (3.42) as a(N-l) and define 11. . (N-l) as 

• m^cn-d .= ^[w 00 - w oo^^ w oo 4 ' +Bl " 1< '’ w oo ],f i 


(3.43) 

(3.44) 

(3.45) 
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{m 


i 


Replacing the time arguments, the above results may now be summarized 


for the single-stage problem. 

J 

u(N-l) « Z S . (N~l)£(N-i-l | N~l) ’ (3. 46) 

i-0 1 

S^(N-l) « -[t|i’(N,N-l)W 00 (K)^(N,N-l) 

+ B(N-l)]“ 1 ^ , (N,N-l)W 00 (N)(|» i (N,N-l) 

i « 0, 1, ... , J (3.47) 

W. (N) « /A(N) i « j « 0 . 

** v 0 elsewhere , (3.48) 

J J 

V. * E{ E Z x' (N“i-l)H i | (N-l)x(N-‘j-l) } + a(N-l) (3.49) 

1 i~0 j=0 4 

a 

\ (N-l) - <^(N,N-1) {W 00 (N)-W 00 (N)t//(N,K-l) ’ (N.N-nW^NWM -i) 

+ B(N-l) ] -1 ip ' (H,N'-1)W 00 (N)}^.<N,N-1) 

«= ^(N,N-l)W 00 (N)'t’ (N.N-1) + ^<N,N-1)W 00 (N) 

• *(N,N-1)S (N-l) (3.53) 

J 

J J 

a (N-l) «= trace { Z Z t A . (N-l)P(N-j-l,N-i-l|N-l) 

1=0 j =0 1 - ) 


+ r'(N,N-l)W 00 (N)r(N,N-l)Q(N-l)) (3.51) 

.^(N-l) = S^(N-1)((-’(N,N-1)W 00 (N)iKN,N- 1) + B(N-1)] -1 S (N-l) 

(3.52) 


where the initial conditions are 


(N+l) = 0, a(N) = 0 


61 


3.5:2 Double-Stage Problem 

~ ' r 1 • • 1 n • L i -i r-m nmn. ^ 

» 

The two-stage problem may now be written 

V 9 * min min E {[x* (N~l)A(N-l)x(N-l) - u’ (N-2)B(N-2)u(N-2) ] 
u(N-2) u(N-l) 

+ [x 1 (N)A(N)x(N) + u* (N-l)B(N-l)u(N-lj) (3.53) 
where the expected value is over x(N), x(N“l), u(N-l), u(N-2), Note 

that u(N-l) and u(N-2) are required to be physically realizable. Util- 

* 

izing the principle of optimality, Eq. (3.53) may be rewritten 

min E{x’(N-l)A(N-l)x(N-l) + u* (N-2)B(N-2)u(N-2) + V.) 

1 u(N-2) . 1 

(3.54) 

From Eq. (3.49) and (3.51) it can be seen that 

J J 

E (V 1 } * E{E[ 2 2 x f (N-i-l)M. . (N-l)x(N-j-l) + ct(N~l]} (3.55) 

i-0 j=0 ^ 

J J 

« E{ Z 2 x 1 (N-i-1 ) M . . (N-l ) x (N~ j -1 ) } v a (N-l) (3.56) 

i=0 j-0 13 

since the inner expectation of Eq. (3.55) is eVer x* 

Thus Eq. (3.54) can be written 

V = min E{x’(N-l)A(N-l)x(N~l) + u* (N-2)B(N-2)u(N-2) 

■ u(N-2) 

J J 

+ 2 2 x* (N-i-l)M. .(N-l)x(N-j-l) } 4- a (N-l) (3.57) 

i=0 j=0 13 

J J 

- min E { 2 2 x* (N-i-l)W. . (N-l)x(N-j-l) 

u(N-2) i=0 j-0 XJ 

+ u’ (N-2)B(N-2)u(N-2) } + a (N-l) (3.58) 

where a (N-l) is taken out of the minimization procedure since its value 

does not depend on u(N-2) and the following definition is used 


'3 


V.} 




FT: 

F • 

; C 

l ,1 


1 


W (IKL) .s 


M ± j (N-l) + A(N-l) 


t - j * 0 


M. . (N-l) 
*3 


85 0,1, >M t J 
(except i « j « 0) 


0 

v_ 


i.j >0 


(3.59) 


Since W and M are symmetric W . . - W: . and M. . = M; This definition is 

ij ji rj ji 


a matter of convenience for later development. Because of the system 
equation (3*14) 


x(k+l) « £ <£ (k+l,k)x(k) + $ (k+l,k)u(k) + r(k+l,k)w(k) (3.60) 


i=0 


u(k) can only affect x(k+l). Therefore, since W.. * W* » the minimiza- 

♦ J J i 


tion of Eq. (3.58) may be rewritten 


V 0 * min E {x ' (N-l)W nn (N-l)x (N-l) 
2 uf.N-2) UU 


J-l 


+ 2 £ x 1 (N-l)W (N-l)x(N-j-2) 


J“0 


0,j+l 


+ u , (N-2)B(N-2)u(N-2)} (3.61) 
Substituting the plant equation (3.60) into Eq. (3.61) and momentarily 
dropping the time indices 


J J 


V = min E{ £ £ xV<^W Q <J>x +. 2u>’W £ <p.x 

4 u(N-2) i=0 j=0 1 1 U J 3 OO i=0 


+ 2w'r’W Art £ 6.x. + 2u’^’W rw + u* * W iM-B]u 
00_._ n 1 1 00 00 


i=0 


00 1 


J-l J 


J-l 


+ w ' r ’W w + 2 .f 0 .f-i^oj^i + 


j-i 

+ 2- £ w'T'W„ . , x . } 


(3.62) 
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As in the development of Eqs. (3.24) and (3.25) for the third, fourth 
and ninth, terms vanish. Because the terms x_^ *=» fc(N-i-2), i = 0, ... , J) 
occur at or before N-2, they do not depend on u(N-2). Thus the minimiza- 
tion of (3.62) depends only on the second, fifth and eighth terms. 

Utilizing the properties of conditional expectation as in equa- 
tions (3.23) through (3.26), Eq. (3.62) becomes 

J J J 

V 9 = min E{E[ E E x*^ lW ftn 4> .x . + 2u'»J> , W 00 E ^x* 

1 u(N-2) i-0 j“0 1 1 00 3 3 i-0 ■ 

+ u , [t*'W nn ^B]“ 1 u + w’r’w rw 


00 
j-i j 


00' 
J-l 


+ 2 jfo t So^H W °.J+i x J * 2 jf 0 u, * ,w 0.j+i x i 


z*(N-2),x*(0)]} 

(3.63) 

Now Eq. C3.63) is minimized if the inner conditional expectation 


E {■ 


z*(N-2) ,x*(0) } is minimized with respect to u(N-2). Since 


x^~ x(N-i-2) then x^ is unaffected by the minimization. .Thus the first 
and fifth terms of Eq. (3.63) are unimportant. Similarly the fourth 
term is unaffected by the choice, of u(n-2). By setting the partial der- 
ivative of the inner expectation (less the first, fourth and fifth terms) 
with respect to u(N-2) to zero 

J J-l 


* V, 


& 


= 0 = E{t{i 


% 0i yi Xi + i*v»* + 


x ,» 4 •> 

Z* ,X~ / 

(3.64) 


By solving for u, this becomes 


u(N-2) = -[•> 1 W 00 .J+B]- 1 ?.’tW 00 Z *** + Z W„ 

i-0 i-0 * 


J-l 


(3.65) 


* 



/{ * . 


*. * 



Define S t <U-2) = -[U^ + W 0>t+1 ] i » 0, ... , J (3.66) 

and recall from Eq.. C3.59) that s 0 if 1 or j > ‘J 

J 

u (N-2) “ E S . (N-2)x(N-i-2 |N-2) 
i-0 1 

V2 may now be evaluated by substituting Eq. (3.60) and Eq. (3.65) into 
Eq. (3,57), omitting the third, fourth and ninth terms which have been 
shown to vanish. 


J-l J-l 

V n *=■ min E{ E E x!W. , , x. + Z Z x * .x . 
2 u(N-2) 1=0 j=0 1 i+1 >j+l j 1=0 j=0 11 00 3 J 


J J 


J J 


" **0 .**i*iW [ ’ ,, »oo M] ‘ 1 ’ , Vj x 

3 


J-l J 


- 2 


i f 0 .f 0 *i w o,i + i^<'’'W +Br ^ , 'W j x j 


j 

J J 


“I,.,, 


+ E E i|)+B] O 00 <M, . 

j[=0 j=0 J J 


J J-l 

+ 2 E E x!«f>!W n . , - x 

1=0 j=0 1 * 0 »J +1 J 

J J-l 


"“3. - 1 , 


+ 2 E E J+B] >'W *. 

1=0 i=0 1 1 00 00 0,j+l 3 


J-l J-l 


1 1 


- 2 j 0 ^ *i* i W oO* I *’ W 00* +Bl ' 1 * ,H Q.j+l x J 


J-l J-l 


-l,.v 


- 2 E E *X n >+B] W a * 

1=0 j=0 1 °» :L+1 00 0,3+1 
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+ w f r!w 00 rw 

+ a(N-l) 


(3.67) 


This definition of W expressed by Eq. (3.59) allows all upper limits on 
the sums to be J. Combining terms, Eq. (3.67) may be rewritten 

' V * “ Jo + UVj + ♦A.i+dV 

I ti 


+ E{&[[- 2 (|>*W 00 ^^'W 00 ^+B] >^00^ 

00' j 


- 2W^ ii+1 *t<p’W 00 *+B] -1 * 'W ♦ 


I « 1 


0,j*KL 


- 2t±u 00 m'\t 00 m] , 

’0,i+l rir 'W ^ J r "0,j+l J ~j 


- 2< 


+ B l+iWoo* [**W 00 *+BJ - 1 * » 


+ 2^H 00 ftf W 00 HB rl * ,W 0 , j+ l ,A j ) . 


+ Efa'T’W Fw) + a (N-\L) 


(3.68) 


Substituting = x^ - 5L in Eq. (3.68) and noting that 

- i- . 

Etx’yx) = tr E{yxx'} - 0 since the term in parentheses is a scalar 



2 " t f 0 jj; 0 ® ix i [W i+X,j+l + *t W 00*J , + V'O.j+l 


- 2 ^V N,, W +Brl ^'Wj 

- 2*iv 00 *[# 1 w 00 t»]“S'w 0ij+1 

- 2 ^,i + ^^’ w 00 ^ B 5‘^' w o, j+ i 

+ w o.i-i* I * ,w oo^ ] ' 1 * ,w o >j+ i 
+ 2*i» 0 0*» , «po* + B]-S'W 0ii+1 J^} 
+ E{xJ_[^.U 00 ^ti>' K 0 ^+B 1 ’ W * 

+ 

+ V B i'V« 0ij+1 iv 


+ E{w»r'W 00 rw} + a(N-l) 


-r, 


Define, Ay - [^K 00 + U i+1 ,0> [ *' K 0 0* +3) • [» 0>j+1 + "ooV 


Substitution from Eq. (3.65) yields 

,Aj = ■ s ii*'w Q o* +B J" ls 1 


The second expectation of Eq. (3.69) may be written as 


4 



* - . v 



J J 

E{.E E x! A. .x .} 
1=0 j=0 1 ^ j 


(3.71) 


Define S W i+1>j+1 + *iVj + \ W 0, j+ l 

-♦iV t *' w oo+ +Brl * ,w oo*j 


- "m.o^ V BrVw o, j+ i 


(3.72) 


Substitution of Eq. (3.66) into Eq. (3.72) yields 

M.. « VL., ... + <j> W n ... 
ij i+l,j+l T i 0,j+l 


+ w ; h 00 + "i+i.oWj + ^s.(k) ] 


J J . 

By letting cc(N-2) = E{ l t xJA.x. + wT'W. Fw + a(N-l) 

i-0 j=0 x 3 U0 

J J • 

- $ X tr(A,P..) + tr(r f W n FQ) + a(N-l) 
. i=0 j=0 iJ 00 

The expression for V_ may now be written from Eq. (3.69) as 

J J 

• V 0 =*E:U l x!M, ,x.) + cx(N-2) 

2 1=0 j=0 1 i; J 3 


(3.73) 


(3.74) 


(3.75) 


The results for the two-stage optimization process may now be summarized 
with the time indices restored in Table 3.1. 
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Table 3.1 


Summary of Optimal Control Algorithm for 2-Stage Sequence 


u(N-2) - £ S (N-2)£(N-i-2| N-2) 


(3.76) 


i-0 


S (N-2) « ~[i|j * (N-l ,N~2)W nn (N-'l)ip (N-l , N-2) + B(N-2) 3"* 1 


^(N-i) , 


J J 


00 

• ^ , (N-l > N-2)[W 00 (N-l)4> i (N-l,N-2) + W 0>i+1 ( N “D ) 

(3.77) 

i = j = 0 

i,j - 0, ... , J except i.~j-0 
0 ' i,j>J ’ (3.78) 


A(N-l) + M (N-l) 
ij 

M tj (N-l) 


V = Ei E I x 1 (N-i-2)M.. (N-2)x(N-j-2) } + a (N-2) 
2 i-0 j=0 3 


(3.79) 


(N-2) « [<{) ^ (N-l , N-2 ) W Q0 (N-l) + W ±+1 (N-l) 3 [<*> . (N-l, N-2) 

+ t|/ (N-l, N-2) (N-2)] -1 ^ 1 (N-1,N-2)W 0J+1 (N-1) 


+ ^4.1 • . i < N ~ 1 > 

1 + 1 , 3 +! 

J J 


(3.80) 


a (N-2) = trace [ E £ A. (N-2)P(N-j-2 ,N-i-2 I N-2) ] 

i-0 j=0 lj 

+ trace [ V 1 (N-l ,N-2 ) W Q0 (N-l) V (N-l ,N-2)Q (N-2) ] 

+ Qt(N-l) (3.81) 

^(N-2) = S^(N-2) [^' (N-l,N-2)W 00 (N-lH(N-l,N-2) 


+ B (N-2) ] "" 1 Sj (N— 2 


(3.82) 
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Despite the complex nature of the various equations^ note that 

the computation for the two-stage process is quite straight forward, 

* • 

First, the feedback gain matrices {S (N-2) , S (N-l) , 1**0 »,*.». , J} are 
* i i 

determined and they are used in conjunction with the optimal filtered 

estimates {£(N-i-2|N-2) ,£(N-i-l|N-l) , i=0, ... , J) respectively, to im 
* 

plement the control signals. Thus the separation principle is again 


apparent. Comparison of Eqs.(3.76) through (3.82) with the results for 

> 

the single-stage case, Eqs. (3.46) through (3.52), the results are seen 
to be identical except for the change in time index. In effect, Eqs. 
(3.76) through (3.82) reduce directly to Eqs, (3.46) through (3.52) by 
recognizing that Wqq(N) is the only non-zero term of W for the single- 
stage case. 


3.5.3 t-1 Stages 

Assume now that the results of the two-stage case may be gener- 
alized to t-1 stages. The equations characterizing the control as ex- 
pressed in Eq. (3.76) through (3.82) then become as in Table 3.2." 

3.5.4 t Stages < 

From the principle of optimality 

V = min E (x ' (N- 1+1 ) A (N-t+1 ) x (N-'t+l ) 
t u(N-t) 

+ u'(N-t)B(N-t)u(N-t) + V t-1 ) (3.90) 

Examine from Eq. (3.86) and express it in a slightly different 

form so that the minimization becomes more evident. 



Tabic 3.2 

■*' 

Summary of Optimal Control Algorithm for a (t-1) -Stage Sequence 


u(N-t-l) « £ S. (N-t-l)*(N-t-i+l|N-t+l) 

i=0 X 


(3.83) 


S^(N-t+l) *= -tyHN-t+2,N-t+l)W 00 (N-t+2)i|j(N-t+2,N-t+l) + BCN-t+l))" 1 

•^(N-t+2,N-t+l)[W 00 (N-t+2)<}. i (N-t+2,N-t+l) 4* W 0>i+1 (N-t+2) ] 

i * 0, 1, ... , J (3.84) 


W ij ( N ~t+2) .*= (N-t+2) + A (N-t+2) 


i ^ 0, 1| ... , J 
i-J-0 


Hy (N-t+2) 


i,j ~ 0* ... , J except i™j=0 




(3.85) 



M^N-t+l) = (N-t+2, N-t+l)W 00 (N-t+2) + \ +1 >0 (N-t+2) ] * j 

• M> (N-t+2 ,N-t+l) + ip (N-t+2, N-t+l)S.(N-t+l) ] j 

+ <|.^(N-t+2,N-t+l)W 0J+;) (N-t+2) + W i+1>j+1 (N-t+2) 

i»j ~ 0 » 1 > • . .• » J (3.86) ^ j 

j 

J J ’ ' I 

V->.i = E IE £ x' (N-t-i+l)M. . (N-t+l)x(N~t-j+l) + a(N-t+l) . (3.87) 

- A i»0 j-0 ^ , ’ * 

J J 

e(N-t+l) = trace[ 21, £ Aj . (N-t+l)P(N-t-j+l,N-t-i+l|N-t+l)] + a (N-t+2) 

i=0 j=0 

* 

+ tracelr’ (N-t+2 > N-t+l)W 00 (N-t+2) T (N-t+2, N-t+l)Q(N-t+l)] 

♦ * * 

(3.88) 

* i 

; A . (N-t+1) - S i (N-t+l)[^ (N-t+l,N-t+2)W 00 (N-t+2 ) i|j (N- t+l f N-t+2) 

*•3 » 

+ B(N-t+l) ] _1 Sj (N-t+1) 

i,j - 0, 1, ... , J (3.89) 

: . ■ . I 

where M , = M!. and A. - A'., and W. . - WJ . ! 

ij ■ jx • ji ij Ji 

‘ ■ • - ■■ "• . : ' ■■■' 7 ■’ ' ” ■ I 

• ■ . •• • j 

. . - i 
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J-l 


■V. « min E(x'(N-t+l)M x(N-t+l) + 2 £ x' (N-t+l)M_ , x(N-t-j) 

W u(N-t) 00 j«0 °»J +1 


J-l J-l 

+ E 5 x r (H-t-l)K ul u1 x(H-t-J)} + «(H-t+l) (3.91) 

t«0 j«0 i+i ,3+1 


Since the expectation is over x, w and u, E {V , } » V , as expressed in 
« , t-1 t-1 

Eq. (3.90). Substitution of Eq. (3.91) into Eq. (3.90) yields 

V t * min fe{x* (N-t-KL) [A + M 00 ]x(N-t+l) + u f (N-t)Bu(N-t) 
u(N-t) 

J-l 

+ 2 E x 1 (N-t+l)M x(N-t-j) 


jeQ 


'0,j+l 


J-l J-l 

+ E E x’(N-t-i)M iX1 .^xOi-t-j)} + a(N-t+l) (3.92) 
i~0 j«0 


t+1, j+1 


Once again, using the properties of conditional expectation, Eq. (3.92) 
becomes 


V « min EfE x' (N-t+1) lA+M_]x(N~t+l) + u* rN-t)Bu(N-t) 
* u(N-t) 00 


+ 2 ^ o x ' (N ' t+i;,M 0l j+l X j + t f Q j f 0 X i M l+l,j+l X j 


z*(N-t), 


x*(0) } + a(N-t+l) 

vhere x^ - x(N-t-i). Expansion of the inner expectation and substitution 
of Eq. (3.14) and (3.85) in Eq. (3.93) yields 

i - j = 0 

i,j = 0, ... , J (except i = j = 0 ) 
elsewhere (3.94) 


W, . 
ij 


r 

M. . + A 
iJ 


r 


*r#« 
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J J 




+ 2w f r l W 00 £ <J> ^x^ + 2 uVU 00 rw + uM^’Wqq^+BJu 
i^O 


J-l J 


J-l 


+ w f r'w nn rw + 21 j: x!<j>!w n , ,.x + 21 u^ f w n , ,_x 
00 t=0 is=0 i y i 0 ,j+l j jK0 y 0 ,j+l j 


J-l 

+ 23 : w e r f w, 

j-0 


0,j+*l X j 


z*(N-t) ,x*(0)} + a(N-t*KL) (3.95) 


As' shown by Eqs. (3.24) and (3.25), the third, fourth, and ninth terms 

v 

Vanish and a(N-t-f-l) does not depend on u(N-fc), The time indices are 


x^ « x(N-t-i) 


- ^(N-t+^N-t) 
i | > « \l>0> l-t+l,N-t) 


W lj ~ W ij < N ~ t+1) 


u = u(N~t) 
w *= w(N-t) 

B « B(N-t-l) 

* * 

Observe that- the first and sixth terms are unaffected by the choice of 

u(N-t) and minimize Eq, (3.95) with respect to u(N-t). 

.f J J 

V 3V - 0 ■= [ 21 ^'W S 4.x, + 2*' £ W. ,,,x 
V • a^ICN-O I °°i=0 1 1 i“0 °-j +1 j 


z* ,x*(0) ] 


+ 2[if,^ 00 i/r+Bju 

By. solving, for u and restoring the time indices 


(3.96) 


u(N-t) = l S. (N-t)£(N-t-i|N-t) 

1=0 1 

Sj^N-t) = -ij)'(N-t+l,N-t)W (S-t+DtKN-t+l.N-t) + B(N-t) ] 


(3.97) 

-1 


• ip' (N-t+1) [Wg-lN-tMH, (N-t+1, N-t) 

+ w o,i+i (N - t+1)3 


(3.98) 
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Now V must be evaluated from Eq. (.3.91) 
t 


V.. « E{x ! (N-t+l)VT x(N r t+l) + u'(N-t)B(N~t)u(N-t) 
t 00 


J-l 

+ 2 E x* (N*-t+l)VL .,.x(N-t-j) 
j«0 u »J +i 


J-i J-l 

+ E E x'CN-t-DW, 
i=0 j=0 


x (N-t- j ) + a(N-t+l)} (3.99) 


i+l,j+l 

Substitution of the system equation (3.14) into Eq. (3.99) as well as the 

expression (3.97) for u(N-t) yields 

* 

j j j j 

v t - E{ £ £ x!*!W * x + 2 £ x!s'*'w £ 4> x 

* 1=0 j=0 1 1 00 J J 1=0 1 1 °?j=0 j j 


J J 


,-l. 


+ E E xJS'lfWj+B] x 
i=, 0 j=0 1 * 00 * ^ j 


+ u'r'\i QO rvj 

j-i j * 


j-i j 


+ 21 E xI^'VT x + 2 E E £*S’tJrW * x. 
■ j=0 i=0 ** ** ®»1^*1 ^ * j=0 i=0 ^ ^ 0, j+l 1 

J-l J-l 

+ E E xl-W. - ’ ...x,} + a (N-t+1) * 

1=0 j=o 1 1+1>J+1 J 


(3.100) 


Once again, since W - 0 for i or j > J, the upper limit on the sums 

i»j 

may be J for each sum without changing the expression (3.100). Therefore, . 

Eq. (3.100) may be rewritten, letting x^ “ ~ an< * noting that 

Efx.x!) = 0 for all 1, j. 
i 3 * J 


t 

i 
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4 k l ^ W ° 0 ^ + 2S ^' W ° 0 ^ + S t [ ^.' V 531 

+ ♦i*b,j + i + S ?' w 0, j+ 1 + w i + l, j+ l *j / 


+ S E SJ sn^'W >HE]“ 1 S, x, 
1=0 j=0 uu J J 

+ E{yWA rt rw 


x 4 ) 


oo 

+ a(N-t+l) 

* . .» 

where the time indices are 


x. *= x(N-t-i) 
• i 

(j» t « ^(N-t+l.N-t) 

S jL « S(N-t) 

„ * 

^ (N-t+1, N-t) 

W. . = w. .(N-t+1) 
ij 3-J 

r « r (N-t+i, N-t) 


w « w(N-t) 


Define t = S^^’W^+B]" 1 ? 

J J 


h trace Z ZA. P, . + r'W n rQ] 
1=0 j=0 13 uu 


a(N-t) 

+ a(N-t+l) 

then Eq. (3.100) becomes 


J J 

EM. Z x!M. .x.} + a(N-t) 
i=0 j-0 1 3 - 3 


(3.101) 


(3.102) 


M i3 S ^i W 00 + W i+l,0 ][ *j + *V + *i W G J+l + W i-l,j+l < 3 ’ 103 > 
J J 


(3.104) 


(3.105) 


But this is the same result as Eq. (3.86) with the time index changed. 

It remains to be demonstrated . that , in fact, the u(N-t) expressed 
by Eqs. (3.94), (3.96) and (3.102) minimizes the performance measure 
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given by Eq. (3.90), Recall from, differential calculus that the van- 
ishing of the gradient with respect to u, which led in the general case 
to Eq. (3,96) is only a necessary condition for V to be a minimum. 

That is, u(N-t) in Eq. (3.97) only guarantees that V. attains a station- 
ary value. A sufficient condition that V attain a minimum is that the 
second gradient of V. with respect to u(N-t) be positive. This condi- 
tion is determined by examining the gradient of Eq, (3,96). 

J J . 

I gt>j “ + + V +B]u i < 3 - 96) 

The "second gradient" of Eq. (3.96) becomes 

V »<N-t) [V u(N-t)< V t>l " *V B 

Thus a sufficient condition that a minimum be obtained is that 
the matrix 

i|> * (N-t~l,N-t)W 00 (N-t)i|> (N-t-l,N~t) 4* B(N-t) 
be positive definite for all t = 1, 2, ... , N. 

The control algorithm and associated performance measure equa- 
tion’s may now be written, letting k = N - t. These results are summar- 
ized in Table 3.3. Block diagrams of the optimal controller and the 
optimal control system are .shown in Figures 3.2 and 3.3 respectively. 

The control law expressed by Eq. (3.105) requires the optimal 
estimate of the delayed states at each stage. The form of the control 
law indicates the following theorem: 













! 
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Table 3.3 

*• 

Summary of; Optimal Control Algorithm, for k~Stage Sequence 


J 

u(k) = Z S.(k)*(k-i|k) 
1=0 


(3.106) 


WjjCk+1) - 


r 


M. . (k+1) + A(k+1) 
* 3 


S i (k) = -[t)'(k+l,k)W 00 (k+l)^(k+l,k) + B(k)] _1 

• *'(k+l,k)[H 00 (k+l)<f. i (k+l,k) + U 0ji+1 (k+l)] 

1 « 0, 1 J (3.107) 

i=j=0 

... » J except i=j=0 

0 i,j>J . (3.108) 

M^OO « U|(k+l,k)W 00 (k+l) + \ +1>0 (k+l)][<|» j Ck+l,k) + ^(k+l # k)S (k)) 

+ ^(k+l,k)W 0>j+1 (k+l) + W i+ljj+1 (k+l) 

i,j «= 0, 1, ... , J (3.109) 


t 


V k+ D 


J J 

V ; K » E{ £ E x* (k-i)M. . (k)x(k-i) } + a(k) (3.110) 

* 1=0- j=0 30 . 

J J 

a(k) « trace { E Z, A. (k)P(k-j ,k-i|k) } + a (k+1) 

’ i=0 j=0 2 

+ trace {T* (k+l,k)W 00 (k+l)r(k+l,k)Q(k)} (3.111) 

• /^(k) - S[(k)[^ , (k+l > k)W 00 (k+l)^(k+l,k) + B(k) ] X S j (k) 

i, j = 0, 1, ... , J (3.112) 

for k - N-l, N-2, ... , 0,. where a(n) = 0, VJqq(N) = A(N) , and VI ^ (N) is 
zero elsewhere. The r x r .matrix [ <p ' (k+1 , k) Wq q (k+1 ) \jj (k+1 , k) + B(k)] is 
required to be positive definite for all k. The minimum value of the 
performance index, for (N-k) stages of control is given by Eq. (3.110; . 
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Theorem 3. 2 The optimal control system for the sto- 
chastic linear regulator problem consists of the 
optimal linear filter cascaded with the optimal . 
feedback gain matrix. The parameters for the two 
parts of the control system are determined inde- 

8 * 

pendently. The performance measure is governed by 
Equations (3.109) through (3.112) where the boundary 
condition is a(N) =0. 

The next section verifies that the gain matrix is the same as 
that for the deterministic controller. This confirms the separation 
principle as applied to time delay systems. 

3.6 Discussion of Results 

» * 

3 . 6 . 1 Comparable Results for Deterministic Case 

• It is interesting to relate the results expressed in Eqs. (3.106) 
through (3.112) to the deterministic case where no plant disturbances or 
measurement errors are present. For such, a case the noise covariance 
matrices, R and Q, are identically zero. The estimates of the states 

V-" 

are equal to the states themselves forcing the filtering error covariance 
matrix to zero and the expected value operation is over a deterministic 
quantity, yielding the quantity itself. The resulting equations (3.106) 
through (3.112) are presented in Table 3.4. 

Note that the computation of the control law expressed by Eqs. 
(3.113) through (3.119) once again verifies the separation principle. 
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Table 3.4 

Summary of Optimal Control Algorithm 
for Deterministic Time Delay Systems 


u(k) « E S.Ck)x(k-i) * (3.113) 

i*=0 


i-l 


S t (k) - -^*(k+l,k)W 00 (k+l)^(k+l,k) + B (k) ) 

4»' (k+l,k) [W 00 (k+l)<j> t (k+l,k) + W <k +1) ] 


W^Ck+l) = 


M (k+1) + A(k+1) 


>I ii (k+1) 


i * 0, 1, ... , J (3.114) 

i c j-0 

i,j-0, ••• » J except i=j=0 
elsewhere (3.115) 


^i j (k) ” k+l,k)W 00 (k+l) + W ±+1 0 (k+l)][^ 1 (k+.l,k) + ^(k+l,k)S j (k)) 


+ *[(k+l,k)W 0 (j+ 1 (k+l) + w i+lij+ 1 (k+i) 

i»j ~ 0, 1» ... > J 

J J 

V N , « S £ x* (k-i)M. . (k)x(k-i) 
t-0 j=0 

a(k)* = 0 

,,A . (k) = don't care 
x/j 


(3.116) 

(3.117) 

(3.118) 

(3.119) 


t 
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That Is, the control law is identical to that expressed by Eqs, (3.106) 
through (3.108) except that x(k-i|k) is replaced by x(k-i). Thus the 

controller in the stochastic case treats the optimal estimate of the 

* 

states as if they were the actual values of the states. 

The value of the performance index is not the same, however. It 
is, as expected, less for the deterministic case than the stochastic case. 

3 . 6.2 Comparable Results for Stochastic No-Delay Case' . 

« * 

If’ no delays are present in the‘ system, the results expressed by 
Eqs. (3.106) through (3.112) should reduce to the standard optimal con- 
trol law. The transformation may be made by letting J c 0 and writing 

the resulting equations which are presented in Table 3.5. These results 

* . 

are, in fact, identical to the standard optimal control results such as 
those obtained by Meditch^ Chapter 9. 

3.7 Computational Aspects of Optimal Controller 

The recursive nature of the computations required to generate 
the optimal control sequence and to evaluate the performance index is 
evident from Eqs. (3.106) through (3.112). 

1. Given (W. . (k+1) , i, j=0,l, . . . , j) compute (S.(k), i=0,l, ... ,J} 

■*-J i 

from Eq. (3.107) . 

■ ♦ • •• * 

2 . Substitute the values of {S^(k), i=0,l, . . . ,J) and WQQ(k+l) 
into Eq. (3.112) to obtain ■ {A. . (k) , i, j=0,l, , , . ,J} . 

3. Substitute {A (k), i, j=0 ,1 , . . . , J} , Wqq(R+ 1), a (k+1), and 

the error covariance matrix {P(k-j ,k-i| k) , i, j-0,1, . . . , J} into Eq. (3.111) 
to obtain a(k) . 



i A 
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Table 3.5 

■# 

Summary o£ Optimal Control Algorithm ior Stochastic Systems 


With No Time Delay 

* 

* 

/* 

u(k) « S 0 (k)S(k|k) ' ' (3.120) 

S 0 (k) - - 1 if ' (k+1 , k) W 0Q (k+1 ) tji (k+1 . k) + B(k)]- 1 

» 

•(■•(k+l.WW (k+l)$ 0 (k+l,k) , . (3.121) 

* • 

W 0Q (k+l) = M (k+1) + A (k+1) ' (3.122) 

Mo 0 (k) = 4> 0 (k+l,k)W 00 (k+l)[(. 0 (k+l,k) + Kk+l,k)S 0 (k)] (3.123) 

V K _ k = E(x'(k)M 00 (k)x(k)) + a(k) (3.124) 

* 

a(k) = a (k+1) + trace {f* (k+l,k)W 0 Q(k+l)rCk+i,k)Q(.k)} 

+ trace (A 0 q ( k) P (k | k) ) . (3.125) 

,A 00 (k) •= S’(k)[J<’(k+l,k)W 00 (k+l)^(k+l,k) + B(k)] _1 S 0 (k) 

» -(.^(k+l,k)U 00 (k+1) (k+1 , k) S Q (k) 


(3.124) 



/ 
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4. {S^(Jk) * 1^0,1, ... ,J) and (k+1) , i,j**0,l, . . , ,J) are sub" 

stituted into Eq, (3.110) to obtain 00 » it j R 0 ,1, . . . ,J) . 

5. Substitute, (k) , i, j»0,i, . . . ,J) into Eq. (3 V 110) to obtain 

W 

6. Substitute {M. . (k) 4i i, j«*0,l, . . . ,J) and A(k) into Eq. (3.108) 

* 

to determine {W^j (k) , i, j=0,l, . . . ,J) . 

•* 

The cycle is then repeated, letting k become k-1 in the above 
procedure. Note that although {M^ (k) , , i, j-0,1, . . . ,J) must be computed 
at each stage, is not necessarily of interest and, therefore, need 

only be computed at k = 0 to obtain the minimum value of the performance 
measure for all N stages of control. The values of a(k) and 
.{ j (k) , i,j=0,l, . ,J} must be computed at each stage, however, because 

. a 

of their inter-relationship and the dependence of on a(k). Note 

also that the error covariance matrix from the optimal filter must be 

♦ 

known at each stage. If only the optimal control is required then steps 
2, 3 and 5 may be omitted. 

The optimal control is physically realizable since it is simply 
a linear transformation on the estimates of the delayed states. The con- 
troller in Fig. 3.2 is a set of time varying gain matrices 
{S, (k), 1=0,1, . . . ,j} . 

1 t 

Since the computations proceed backward in time, it is clear 
that the time history of (S^(k) , i=0,l,...,j} must be determined prior 
to system operation. That is, it must be precomputed and stored for 
later use. 
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CHAPTER IV 

jt 

v EXPANDED STATE REPRESENTATION AND DUALITY'" 

4.1 Introduction 

In this chapter an alternate representation of discrete-time 

i , 

systems with time delay is formulated. The system equations (2.6) 'and 
(2.16) are imbedded in an "expanded state" form. This representation, 
discussed in detail in Section 4.2, expands the state dimension of the 
system In direct proportion to the magnitude of the time delay. Once 
the time delay system is expressed in the expanded state form the solu- 

, J c 

*tions are well-known * since this form does not express explicitly 

the time delay dependence. Unfortunately, since the resulting system 
dimension is directly proportional to the time delay magnitude, the re- 
sulting computation required for a solution is often considered too ex- 
tensive for practical application; As a consequence the expanded state 

wtr * 

form does not receive much attention in the literature. 

The major results of this chapter are obtained using the expanded 

* ■ ♦ 

'State formulation. One consequence of casting the time delay problem in 
the expanded state form is that the resulting matrices have a large num- 
ber of null elements. Thus, rather than work with expanded matrices (as 
previous authors have done) , it may be more efficient to partition the 
matrices. The solutions to these submatrix equations are obtained in 
Sections 4.3 and 4.4. The results are identical to those of Chapters 2 
and 3. In Chapter 5 the computational savings are demonstrated, by 

84 
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comparing the partitioned results to those obtained by working with the 

*. 

entire matrices. 

In Section 4.5 the expanded state form is used to, demonstrate 
the duality of estimation and control for time delay systems. Duality 

i 

follows as a natural consequence of being able to express a time delay 

. . * 

system as an expanded system with no time delays. It is also shown, how 

ever, that duality can not be extended to the partitioned solutions to 

*■ 

the estimation and control problems. 

•* 

4.2 Expanded State Representation of Systems with Time Delay 
The system equations of Chapters 2 and 3 are 

* 

J 

x(k-KL) « E $ . (k*M,k)x(k~i) + ip (k+l,k)u(k) 4* r(k+l,k)w(k) (4.1) 
i*=0 

z(k4-l) * H(k+l)x(k+l) 4* v(k4*l) (4.2) 


These equations may be also written as an expanded state representation 
x(k4*l) « <|> (k+1 ,k)x(k) + x(k4-l,k)u(k) + r(k+l,k)w(k) (4.3) 

z(k+l) = J((k+l)x(k4-l) + v (k+1) • (4.4) 

where the following definitions apply 


$(krfi) *= x(k+l) 
x(k) 


x (k-J+1) 
u(k) = u(k) 


(4.5) 


(4.6) 






/ 

« 


x {k-M , k) »• (k+1 , k)1 

0 


86 


H (k+l ) 

$ 

?(k+l,k) 


«* ^(k-f-1) 


0 ... 0. 


<f> 0 (k+X , k) (J» 1 Ck+X,k) 

I 0 


I 0 
0 I 


?Ck+l,k) 


r(k+l,k)‘ 

0 


, *j<k+l,k) 


0 

i 

0 


.1 o 


0 I 


(A. 7) 
(A. 8) 


(A. 9) 


v(k+l) = v(k4-l) 
w(k) a w(k) 


(A. 10) 
(A. 11) 
(A. 12) 


From the previous properties of the smaller matrices 
3£ « n(J+l) vector (state) 
u = r vector (control) 
ft = p vector (disturbance) ' 

£ *= m(J+l) vector (measurement) 
v « m vector (measurement error) 

•c: ; f\j - 

' <J» a n(J+l) x n(J+l) state transition matrix 

'V/ 

F *= n(J+l) x p disturbance transition matrix 


87 


* «{/ ** n(J+l) x r(J*KL) control transition, matrix 

, * 

H ** mCJ+1) x n(J+l) measurement matrix 

* * ,* 

. / 

xCO) « zero mean gaussian random n(J*fl) vector with positive 
semidefinite covariance matrix P(0) 

* *V ■ 

fw(k) , k=0,l,.*»} « zero mean gaussian white sequence which is 

independent of x(0) and has a positive semidefinite p x p 
covariance matrix Q(k), k-0,1,... 

IvCk+l) ,k=0,l, . . .} « zero mean gaussian white sequence which is 

independent of x(0) and (w(k) , k-0,1,...} and has a posi- 

* , 

tive semidefinite m x m covariance matrix R(k-KL) , k+0,1,.. 

. , t 

lu(k), k^O , 1 , . . . } = control sequence which is either known or can 
be specified as desired 

4.3 Optimal Estimation Solution of Expanded State Representation 

With the system described by Eqs. (4.3) and (4.4) having the 
properties expressed in the preceding section, the solution is well- 
known (Meditch ** Chapter 4) for this is the standard estimation 
problem. Once again, of course, the system must be ’’observable" as 
described in Appendix A or the estimation problem cannot be solved. 

Under the assumption of observability, then, the following theorems 
from Meditch , (pp . 174-177 and p. 356) are stated for a system 
described by Eqs. (4.3) and (4.4). 
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Theorem 4.1 If the optimal filtered estimate x(k|k) 
and the covariance matrix • P(k| k) A of the corresponding 
filtering error &(k|k) = 3z(k) - &(k|k) are known for 
some k » 0, 1, ... , then 

a. The single-stage optimal predicted esti- 
mate for all admissible loss functions is given by 
the expression 

* * 

it (k+1 1 k) = $(k+l,k)$(k|k) + $(k+l,k)v(k) C4-*3) . 


b. The stochastic process {ft(k+l|k), 
k ~ 0,^1, ...} defined by the single-stage prediction 
error* & (k+1 |k) « &(k+l) - x(k+l|k) is a zero mean 
Gauss-Markov sequence whose covariance matrix is 
given, by the relation 

$*(k+l J k) = $ (k+1 , k) (k | k) $ 1 (k+1 , k ) 

+ t'(k+l,k)$Ck)?\(k+l,k) (4 .14) 


Theorem 4.2 • 

a. The optimal filtered estimate x (k+1 | k+1) 
is given by the recursive relation 


$(k.+l|k+l) = &(k+l|k) - fc(k+l) [z(k+l) - fe(k+l)$(k+l|k)] 

. , 15) 

for k = 0, 1, ... , • where x(0[0) = 0 


b. • k(k+l) is an n(J+l) x m matrix which is 
specified by the set of relations 

k(k+l) = £(k+l[k)ft’ (k+1) [ft Ck.+l)^CkH-l [ k>ft* (k+1) +K(k+1)]“ 1 

(4.16) 

$*(k+l|k) = <)> (k+l,k)^(k[ k)<|> 1 (k+l,k) + ? (k+1 ,k)?J (k>? ‘ (k+1 , k) 

(4.17) 

£(k+l|k+l) = [I-&(k+l)&(k+l) ]^(k+l[k) (4.18) 

for k = 0, 1, ... where I is the n(j+l) x n(J+l) identity 
matrix and !*(0|0) - 2?(0) is the initial condition for 
Eq. (4.17). 

c. The stochastic process (k+1 J k+1) , k=0 , 1 . . . . } 
wliich is defined by the filtering error relation 

x (k+1 | k+1) = x (k+1) - $ (k+1 | k+1) (4.19) 

k=0, 1, ... , is a zero mean Gauss-Markov Sequence 
whose covariance matrix is given by Eq. (4.18). 


s . 


„■ t,'*' 

r 2* 



/ 

j 


For ‘these theorems, the following definitions hold 

* ■* » ( 

Hk+l| k) ~ E {!x (k+1 1 k)%’ Ck+lk)} 
where P(k+l|k) Is n(J+l) x n(J+l) 

S(k+l|k) = x(k+l|k) 

*Ck|k) 

♦ 

* * 

* • 

*(k-J+l|k£ 
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(4.20) 


(4.21) 


where k(k+l|k) Is n(J+l) x 1 . 

The use of Eq. (4.20) allows £(k+l|k) to be expressed in terms of its 
submatrices as 


J* (k+1 1 k) « 


mm 

P(k+l,k+l|k) P(k-KL,k|k) . . . P(k+l,k-J*H.| k) 
P(k,k+l|k) P(k,k|k) 

• r * * • 

• • • 

[p(k-J+l ,k+l | k) P (k-J+1 ,k| k) , , .P(k-J+l,k-J+l | k) 


(4.22) 


Similarly , 


£(k|k) = 


P(k,k|k) P(k,k-l|k) P(k,k-j|k) 

P(k-1 ,k| k) P(k-l,k-l|k) 


P(k-J ,k| k) 


P(k-J,krj|k) 


(4.23) 


4.3.1 Partitioned Representation of the Expanded State Solution to 
the Estimation Problem 


The solution to the optimal estimation problem for systems with 
time delay is expressed by Eqs. (4.13) through (4.18). As mentioned 
earlier, however, the dimensions of the matrices involved in this expanded 
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state solution may render the computation prohibitive. The appearance 
of a large number of null elements in the defining equations for these 
matrices [Eqs. (4.5) - (4.12), (.4.22), (4.23)] indicate that partitioned 
solutions may be advantageous. Thir? is, in fact, the case as Eqs'. (4.13) 
through (4.18) are examined below in partitioned form. 

Eq. (4.13) may be expressed in partitioned form as 

u(k) 


(4.24) 

The expansion of Eq. (4.24) in partitioned form yields 

J 

x(k+l|k) = z <J> (k+l,k)£(k-i|k) + ^(k+l,k)u(k) (4.25) 

i«0 1 • 

t 

which is identical to Eq. (2.66) obtained in Chapter 2. 


x(k+l|k) 

xCklk) 

• 

• 

• 


♦o ^l • • • 

I 0 '. . .0 

0 I 

• 

• • • 

x(k|k) 

*<k-l|k) 

• 

• 

• 

i 

0 

m 

• 

• 

• 

X(k+1-J|k) 


• . 

• 

0 . . 0 *1 0_ 

• 

it (k- J | k) 

* 0 

0 


Eq. (4.15) may be expressed in partitioned form as 


it (k+1 1 k+1) 

it (k+1 1 k) 

V 

M-.HO ... 0 4 it (k+1 ! k) 

it(k|k+l) 

• • 

^ • 
vy 

II 

- K i 

• 

*(k|k) 

• 

• • 

• • 

5t (k+l-J | k+1 ) 

• 

• 

it (k+l~J | k) 

• 

• 

K j. 

• 

• 

it (k+l-J |k) 

•n 


*= it (k+1 1 k) ~ 1 

K (k+1) [ 

z(k+l) - H(k+l)it(k+l| k) ] 1 


S(k+1-J|k) - K J (k+l)[z(k+l) - H(k+l)S(k+l-|k)]j 

(4.26) 
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The partitioned result of Eq. (4.26) may be expressed as 

*(k+l-i|k+l) « St (k+l-i.| k) - ^(k+l) [z(k+l) - U(k+l)k(k+l|k)] 

0 

(4.27) 

which is identical to Eq. (3.90) obtained in Chapter 2. 

Eq. (4.16) may be expressed in partitioned form 


KpOc+1) 

P 00‘ * * P 0J 



.H O ... 0, 

p oo • • • P 0J 

H'" 

r- 

+ R 

l^Ck+1) “ 

• 

• • 

• • 

0 

• 


\ 

• • 

• • 

0 

• 


• 

• 

K (k+1) 

J 

• •' 
pi p 

L\jo **jjJ 

• 

• 

• 

0 


r 

• * • 

pi pi 

L\j0 • • • \jjJ 

• 

0 

t , 


p oo H ' 

P 10 H ' 


P J0 H ’j 


[HPJ 0 H' + R]" 1 


where P 


ij 

H 


E {x(k+l-ijk)x' (k+l-j |k)} = P fc+l-i, k+l-j |k> 
H(k+1) • 

R(k+1) 


(4.28) 


Restoring the time indices to Eq. (4.28) the partitioned gains become 
K (k+1) « P(k+l-i,k+l | k)H* (k+1) [H(k+l)P (k+1 ,k+l j k)H’ (k+1) 

+ R(k+1)]“ 1 (4.29) 

which is identical to Eq. (2.100) obtained in Chapter 2. 

To ease notation below, the following time indices are used 


P* . - P (k+l-i, k+l-j |k) 
p i,i = PCk-i,k-j Ik) 



Si * 


Examination of th& partitioned form, of Eq^. *29) yields 


3 

00 

3 

10 


p 3 

L , JO' 


01 * 


« • • I 


OJ 


JJ 


< T ) 0 ^’1 


I 

0 


0 . 


0 


0 . . . 0 I 0 


00 


. p 


pi 

L P J0 


1 1 

OJ 
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<|>Q I 0. , .0 

<4 o i 


0 

1 


0 . . . 0 


+ r 


Q,r' o . . . o. 


jj j 1 *‘J i 

21 £ Ztfl o • • • * K V t 3-1 

i=0 j=0 J 1=0 X ■ 1=0 x X \ J X 


2 P n *^4 * * * P 1 1 • • • • 

i=0 °» 1 1 liL 


l,J-i 


J^J-l,^! %• ** P J-1,1 


J-1,J-1 


rqr’ o ... o 

o o 


• • • 


0 

(4.30) 

Restoring the time indices, the following submatrix relationships result 
from Eq. (4.30) 

J J 

P(k+l,k+l|k) = £ E «f) i (k+l,k)P(k,k|k)4>!(k+l,k) 

1=0 j=0 f 

+ r(k+l,k)Q(k)r* (k+l,k) (4.31) 


P(k-j ,k+l|k) = E P (k- j , k-i 1 k) $ ! (k+1 , k) j=0,l, . . . , J-l .(4.32) 

1=0 


PCk+l,k-j|k) = P'(k-j,k+l|k) 


(4.33) 
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i 


Expressions (4.31) and (4.32) correspond to (2.108) and (2,110) respec- 
tively'. 

* 

v Equation (4.18) is now examined in terms of. its submatrices, 

* 

dropping the time indices 


pi pi 

*0,0 * * * r Q,J 


xo ... o’ 

• 


K o 

HO ... 0 

p 3 p3 

*00 * ’ ’ JJ 

a • 

• a 


ox. 

• • 


K i 

a 


. • 

a a 

• • 
)J,0 ‘ • • Pj.J, 


a * 

« • 

0 ... 01 

■■ m 



• 

a • 

. P J,0 • • • P J,J. 


X - K 0 H 


I 

0 


- KH 


0 


I 0 .. 0 


-K H '01 

2 • 

| • 

• I • 

• la 

i 

i °° 


0 

o I 


r p 3 

00 


10 


01 


• a a a 


0J 


P 3 
L JO 


• • • f t 


jj j 


[I - K 0 H]P^ 0 ; [X - K 0 H]P^| ... . . ! [I - K 0 H)P J 



K J W 00 j P J1 


k j hp ox! 


K J HP 0J 


(4.34) 


Restoring the time indices to Eq. (4.26) the submatrices of P(kjk) may 
be expressed as 

P (k-i+1 ,k- j+1 1 k+1) = P (k+l-i ,k+l-j | k) - K i (k+l)H(k+l)P(k+l,k+l~j |k) 


(4.35) 



which. Is Identical to the result {Eq. (2# 111),] obtained earlier using 
properties pf conditional expectation. 

In summary, then, the results of Chapter 2 expressed by Eqs. 
(2.86), (2.61), (2.108), (2.109), (2.110), and (2.111) have been obtained 
in this section as shown by expressions (4*25), (4.27), (4.29), (4.31), 
(4.32) and (4.35) respectively. 


4.4 Submatrix Representation of Expanded State Solution to Optimal 
Control Problem 

, The optimal control solution is well-known (Meditch 5 , Chap- 

ter 9) for the system described by Eqs. (4.3) - (4.12), for this is the 
formulation of the standard discrete-time optimal control problem. Once 
again, of course, the system must be "controllable” as described in 
Appendix A or the control problem cannot be solved. The familiar optimal 
control solution is stated below in Theorem 4.3* This is followed by 
examining the submatrices of the solution equations. These submatrix 
results are identical to the optimal control solution obtained in Chap- 
ter 3. 

If the system is controllable the following results from Meditch, 
p. 356, can be stated. Recall that the stochastic linear regulator 
problem is that of minimizing the Performance Index expressed as 


P.I. = E [x* (i)X(i)x(i) + u’ (i-l)B(i-l)u(i-l) ] 
i=l 

where x and u are defined by Eqs. (4.5), (4.6) and 


(4.36) 


A(k) = 


A(k) 0 ... 0 

0 

» • 

0 . o 


n(J+l) x n(J+l) 


(4.37) 



*' 




* 
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fe(k> « BOO,. x r . .W.38) 

* 

Theorem 4.3 , For a system described by Eqs, (4.3) 
and (4.4) the optimal control system Cor the stochas- 
tic linear regulator consists of the optimal linear 
filter cascaded with, the optimal feedback gain matrix 
of the deterministic linear regulator. The parameters 
for the two parts of the control system are determined 
separately. The performance measure for the complete 
control system is governed by 

* 

V N-R K jfi£x* (k)H(k)x(k)} + a(k) and (4.39) 

« * * * » 

a(k) *» a(k+l) + tr[r’ (k+l,k)ft(k+l)?(k+l*k)?j(k)] 

r * * f * 

- tr $ (k+1 ,k)ft <k+l)ft (k+1 ,k)ft (k)P <k| k) ] (4 . 40) 

where the boundary condition is a‘(N) *= 0 

Theorem 4 . 4 The optimal control law for the stochastic 

linear regulator problem' is the linear feedback control law 

* * » 

u(k) - S(k)x(k[k) • (4.41) 

where the r x n(J+l) feedback control matrix £>(k) is 
determined recursively from the set of relations 

ft (k+1) = M(k+1) + A (k+1) ^ (4.42) 

2f(k) = (k+l,k)W(k+l)^(k+l»k) + B(k+1)]“^ 

1 (k+1 , k ) ft (k+1 ) ^ (k+1 , k) (4.43). 

M(k) * O' (k+1, k) ft (k+1) $ (k+1, k) + (k+l,k)ft(k+l)^(k+l,k)S(k) 

(4.44) 

t ... M 

The procedure is now to express Eqs. (4.39) through (4.44) in 
terms of their submatrices to obtain the results of Chapter 3. First 
the following definitions are made to enable a relationship to be estab- 
lished between the results pf Chapter 3 and those obtained below. In 

- v -. 

the performance measure, Eq. (4.39) , (k) is associated with the state 

' Av : . it' ’ " •■•••• •' ... . .. .. 

vector x(k-i) and x(k-j) of the form 
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x * (kri)M (k)x (k-j ) 

so that the. following definition holds 
H(k) | M 00 (k) .... M 0j (k) 


(4.45) 


l M j 0 (k) Wj j (k)j 


(4.46) 

Similarly, in Eq. (4.40), w(kd-l) may be expressed in partitioned form as 


St(k+1) = 


w 00 (k+i) . . , .w oj (k+i) 


LWjO^+1) 


W (k+1) 
JJ - 


(4.47) 


where W(k+1) is n(J+l) x n(J+l) and W. .(k+1) is n x n. 

‘ J 

Finally, the feedback control gains in Eq. (4.41) may be expressed in 

♦ 

partitioned form as 


3(k) 3 S 0 (k) j S^k)! . . ■ j Sj(k) 


(4.48) 


where S(k) is r x n(J+l) and S^(k) is r x n. From Eq. (4.41) it can be 

* • * 

seen that S^(k) is the gain multiplying the estimated state vector 

fcCk-ijk). Equation (4.41) may nrpw be expressed in- partitioned form as 

u(k) - S (k)j . . . jS.(k) x(k) 
l_° I i J i 

x(k-i) 


x(k-J) 


~ l. S. (k)x(k-i) 
i=0 


(4.49) 


This is identical to the result expressed by Eq. (3.10$) . 
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Eq. (.4.42 ) may be expressed in. partitioned form,* using definitions 
<M6). (.4.47) and (4.37). 


W 00^ C+1 ) * * ’ * 


M 00 (k+1) . . . M 0J <k+l) 


A(k*M) 0 ... 0 
’0 

• ft 

• • 


• • 

• • 

+ 

• 

ft 

• 

• 

WjQ(k+l) . . . Wjj(k+1)^ 


H J0 (k+l) . . . MjjCkrKO 


t 

o • 

• 

• 

• 

» 

i2J_ 


The partitioned equivalent of this equation is 

r 


W^Ck+l) « 


M i;j (k.+1) + A(k+1) i*j«0 


(k+1) 


1» . . . , J except i~j=0 
0 i,j>J (4.50) 

which is identical to the result -expressed by Eq. (3.108) . 

To examine Eq. (4.43) first express the inverse of Eq. (4.43) in 
partitioned form by dropping the time indices. 

tf'SJtfr 1 r - - r - -1 1.-1 -i 



- 



m 


t= 

4 1|> 1 0 . . . 0j 

w oo 

. . . W 0J 

¥ 

+ B 

, , 


• 

ft 

♦ ft 

• 

0 

ft 



• • 

_«J0 

• 

• • • W JJ. 

ft 

0 

» 

• 


l* , 'W + B)“ 1 





) may be written 





nn^rfy' 0 . . 

. 0, 

w oo • ’ • 
• 

W oj 

.0 

4- 6 

0 y l 


* 


• 

ft 

X 0 


* 


• ■ 

ft 

0 I 




• 

Wjo • • ■ 

w jj. 

_0 . 


(4.51) 


♦ 


0 10 


(«’w 00 «+b]-H' W 00 v + w 01 }- 

L__ i 


! w oo*j-i + w oj! w oo+j 

J 1 j 


(4.52) 
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When thn time indices are restored to Rq,> -52) the following results 

.+ * 

ara obtained. 

S t (k) ■= ty , (k+l,k)W 00 Ck+l)i|p (k+l,k) + BCk.+l)] -1 

.*'<fcH,k)[ff 00 (k+lH 1 (k+l 1 k) + W 0it+1 (k+l)] (4.53) 

Recall that Wq =* 0 from Eq. (4.J50). Thus Eq. (4.53) is identical to 

* 

m , 

Eq. (3.107). ,Eq. (4.44) may be expressed in partitioned form as 


Mqo • • • M 0J 


k 1 c 

> . . . 0 


• 

o 

o 

• w o7 

• • 


• 

• 



• 

m • 


*1° 

0 


• 

• 

• • 


• 

• 


• 

• 

9 9 


» • 

. I 


• 

• 

*-> 

‘sP 

• 

• 

• 

O 

• ^ 


l*th 

<-* - • 

O • 

. . . 0_ 


-*jo • 

• W JJ- 

4>o * • • 4>j 

*+ ^ 

to 

o 

• 

• 

* 

J J 


I 0 ... 0 

0 




0 

• 




• • 

• 



• 

• • 

• 




0 . .010 





mm * 

** 




r* 



mm* 

f— 


♦o«00 W 10 «;Vll • • • *>0,J +W 1,J 
*> 00 +W 20 


* J-1 W 00 +U JO 


*>'00 


♦jV 


4'0+1'S 


<P T+'P 8 


0 * ' ' Y J' r “J 

I 0 .... . 0 

0 


♦ * w n +w TT 

J-l 0J JJ 

I I 

0 . . . 0X0 

(4.54) 

When Eq. (4.54) is completely expanded and the time indices restored, 
the partitioned result for ^ (k+1) becomes 

(k+1) = f^'Ckfl,k)W 00 Ck+l> + W i+ i j0 (k+l)][^(k+l,k) 

4 + ip(k+i,k)s j (k)} + <j)|(k+i,k)w 0>j+1 (k+i) 

+ K i+l,j+l (k+1) ; 


(4.55) 


tfl' t 




which, is identical to the result of Eq. (3*109). It should be noted, 

n 

* « i 

of course, that « 0 if i or j > J. . 

* Next expand Eq. (A. 39) in partitional form using the following 


definition 


x^ s x(k-i) 

Vk " i£ , x 0 • • • x t, f M 00 * • ‘ M 0J 


x Q } + a(k) 


L m jo • • • m jjJ x jJ 

• . * * 

j j . 

* E{ I E xlM. .x, } + a(k) * (A. 56; 

i-0 j<=0 x 13 K . 

When the time indices are restored to Eq. (A. 56) it becomes 

J J 

V NMc = E{ i t x' (k-i)M. . (k)x(k-j)} + a(k) (4.57! 

i=0 j=0 13 

which is identical to the result obtained in Eq. (3.110) assuming ct(k) 
is the same. * * 

Finally, examine ct (k) of Eq. (A. AO) dropping time indices 

* f\j 'W.'V 'X/ VWO . - , 

a(k) .If tr(r'wrq) - tr(<J>'W^SP) + a(k+l) (A. 58, 

m. 

Examine the first* term of Eq. (A. 58) in partitioned form 

r 0 • • • °. r»oo • • • "ml rl Tol- r'w nn rQ 0.59: 


Wjo * * * WjjJ 0 


Examine the second term of Eq. (4.58) in partitioned form 



\ 
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♦J 1 0 • ■ 
♦i 01 


• • 

• • 


.♦j° 


0 

I 

0 


JS. 

0 

0 

• 

• 

• »w* 

4> 

0 

. s o • • • s j, 

p oo * * * P 0J I 

• 

• 

• 

• 

• 

- w jo • • 

• 

*5. . . . 

1 

i 

>•»•«» 0 

. 

r 

ni 

..... . 

0 

• *\ 

• 

• 

t-» 

1 


K x 0 

0 I 




0 

I 


w oo* 


J W JO*J 


I*0»Q 0 + V* 
l$i w oo + ''20^ 


I*?oo + V* 

*J W 0O 




• • • 




1 S 1 P 1J 
. J«0 J 3J 


J 


I J 

• ! * s i p u 
1 ] . 


(4.60) 


Let £ denote the ijth n x n submatrix of Eq. (4.60). 

' ‘ . J 

«ij “ I*?oo + W i+l.ol* , WtJ 

Xf— U 

The trace of the total matrix involves only and 


(4 . 61) 


trace 1 £.. = trace { l l [ c> * W on + W . +1 JfS P..} 
1^0 i =0 j =0 1 X * 3 3 


(4.62) 


By substituting Eqs. (4.59) and (4.6.2) into (4.58) and restoring the 
time indices, Eq. (4.58) becomes 
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*00 «*CM-1) + tr{r f (k+l,k)w(k+l)r(k+l,k)Q(k)} 

* t * 

'■■3 3 

- trU Z [ 4 ! (k.+l , k.) W qA ( k.+l ) + V ft (k+l)] 

i=o j=o 1 Ma *”, 

'■* * f 

• ^ (k+1 , k) S . (k) P (k- j , k-i | k) } (.4.63) 

which, is identical to. Eq. (3.111) . * 

* In summary, then, the solution to the optimal control problem is 
determined in this section using an alternate system representation, 

m ' **" .... 

namely, an expanded state representation. The partitioned results are 
identical to those obtained using dynamic programming in Chapter 3 and 
the correspondence is: 


This section 

Chapter 3 

Eq. (4.49) 

Eq. (3.10 6) 

Eq. (4.50) 

■» 

Eq. (3.108) 

Eq*. (4.53) 

Eq. (3.107) 

Eq. (4.55) 

Eq. (3.109) 

Eq. (4.57) 

'Eq. (3.H0) 

Eq. (4.63) 

Eq. (3.111) 


4.5 Duality o_f Estimation and Control in Systems with Time Delay 
For discrete-time linear systems with no time delay, Kalman 
observed that a "dual" relationship exists between the solution to the 
optimal estimation problem and the solution to the optimal control prob- 
lem. His results are presented in the paragraphs below. Section 4.2 
demonstrates that time delay systems may be expressed equivalently in 
an expanded state form. Kalman’s results apply directly to time delay 
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systems in this form arid the optimal estimation and control solutions to 

such, systems are duals. 

* » 

Kalman's notation is retained for this discussion / since the vari~ 

* t 

■ ■ ■ » . « 

ables in the notation of this paper are not duals, whereas those of Kalman 

are. The relationship between Kalman's notation and that of this paper 
* * 
is also presented for reference. 

Since the partitioned solutions to the estimation and control 

problems yielded significant results, the question arises as to the 

duality of these partitioned solutions. It is demonstrated that the 

principle of duality does not extend to the partitioned solutions of the 

optimal estimation and control problems, 

4.5.1 K alman ' s Dual Transformation 

Consider the estimation and control problems presented below 

♦ t 

Optimal Estimation Problem . Consider the dynamic system described by 
Eq. (4.64) and (4*65) 

*(k+l) = ?(k+l,Wlife5 + $(k+l,k)u(k) +T(k+l,k)vHk) (4.64) 

z(k+l) » ft(k+lj|f>.+l) + v(k+l) ’ (4.65) 

where these equations have all the properties described in Section 4.2. 
Given the observed values z(0) ,z (1) , . . . ,zf(N-l) find an estimate 

A 

x(k+l|k+l) which minimizes the expected loss 

N-l - 

J N *=■ E £ E [x(k+l) - x (k+1 1 k+1 ) ] [ x (k+1 ) - x(k+l|k+l) ] ' } (4.66) 

k=0 

: i 5 . ... ' . ' 

Optimal. Control Problem . Consider the dynamic system described by Eqs. 
(4.64) and (4.65) where these equations have all the properties described 


* ... 


* - ' ' * “ k ' * 


\:T 4 i C , 4 «. *4r\ *• * 4k. J . ft.; JL * .*• *z* 


.... i- 
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In Section. A. 2. Given, any state x(k) , determine a sequence 

r * * • * 

in(k) juCkfi) > , . . ,u(k+T~l) } of control vectors which minimizes the per- 
formance index - - 


k+T ; 


/ , 


VIx(K),uCk)] » E{ E tx'Ci)XCi)xCl) + u' Ct-l)B(t-l)u(i-l) ] } 

i^k 

CA.67) 

The recursive relations which express the solutions to these 
problems are expressed by Kalman as: 

Estimation Problem 

A*Ct) - (J»Ct+l l t)P*CtW Ct)lM(t)P*Ct)M'Ct) + RCt)]” 1 (A. 68) 

4»*(t+l,t) - 4>Ct+l, t> - A*Ct)M(t) tlt 0 (A. 69) 

P*(t+1) « <j»*(t+l,t)P*(t)c|» , Ct+l,t) + QCt) (A. 70) 


Control Problem 


-1a 


**Ct) [M'(t)P*Ct)M(t) + R(t)J M'Ct)P*Ct)*(tH,t) (4.71) 

*Ct+l;t) « ict+l.t)'- M4*(t) • t -?T (4.72) 

/ P*(t-1) * J'(t+l,t)P*(t)J*(t+l,t) + Q(t) (4.73) 

where the correspondence between .Kalman’s notation and that of this 
paper is given in Table A.l. 

„ i 

The principle of duality states that if the form of the solution 
to the optimal estimation problem is known then the form of the solution 
to the optimal control problem may be obtained by a simple transformation 
on the solution to the optimal estimation problem. The converse is also 
true. Kalman's Duality Theorem may now be stated. • 


w 


* V 
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# 

*?*■* «*.'• .*s-. * ■ •.**pfc*vr» 


* • . «*,' 


Table 4.1 


Correspondence between Kalman Notation and Allgaler Notation 


Estimation 


Kalman 

J 

r 

Allgaler 

M(t) 

$(k+l,k)K(k+l) 

4*(t+l,t) 

$ (k+1 , k)$ (k+1 | k+1) P" 1 (k+1 | k) 

P*(t+1) 

$(k+l|k) 

M(t) 

* ft < k > 

RCt) 

lt(k) 

QCt) 

(k+1 ,k)Q(k) r (k+1 ,k) 


Control 


M(t) 

i*ct+i,t> 

p*(t-'l) 

M(t) 

RCt) 

QCt) 



3>’ (k+l,k)w(k+l)?l(k) 
&<k) 

$(k+l ,k) 

B(k> 


X(k) 
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Theorem 4.5. The optimal estimation problem and the 

4 

optimal control problem are duals of each, other in 
the following sense. Let t 0. Replace every mar 
trix F(t) ** F(t 0 +x) in Eqs. (4.68) through (4.70) by 
F 1 (t) = F’(T~t). Then one .has Eqs. (4.71) through 

i 

(4.73). Conversely, replace every matrix 
F(t) =■ F(T-t) in Eqs.' (4.71) through (4.73) by 

F’ (t) - F'O^+t). Then one has Eqs. (4.68) through 

» 

(4.70) where the quantities described by Eqs. (4.68) 
through (4.73) are presented in Table 4.2. 


The question now arises as to whether the same theorem yields a 
dual result when applied to the subraatrices . Consider, for example, the 

A 

filter gain matrix, A*(t), and the control gain matrix, A*(t), and make 

the following .definition 

A*(t) « A*(t) 

A*(t) 

2 


A*Ct) 


» 


A*(t) « A*(t) | A.* (t) ! . . . ! A*(t) 

1 z i J 

I 1 : I I I 


The problem is now that of .determining whether the transformation of 
Theorem 4.5 applied to A£(t) will yield A*(t) . Define the following 


matrices in terms of the submatrices described earlier. 



e 

* * 


k. 


* 



e. 
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Table 4.2 

Relationship Between Estimation, and Control Variables 

In Kalman Notation / 

» t 


Estimation 

1. x(t) (unobservable) state vari- 
ables of random processes 

2. y(t) observed random variables 

3. tQ first observation 

4 . <J> (tQ-H+1 , tg+x ) transition 
matrix 

5. P*(tg+x) covariance of esti- 
mation error 

6. A*(tg+ ) weighting of obser- 
vation for optimal estimation 

7. <{)*(tp+T+l , tg+i) transition 
matrix for optimal estimation 
error 

8. M(tg+T) observation matrix 

9. Q(tg-f-T) covariance matrix of 
plant noise 

10. R(tg+x) covariance matrix of 
measurement noise 


Control 

x(t) (observable) state variables 
of plant to be regulated 

u(t) control variables 

T last control action 

$(T-t+1,T-t) transition matrix 

a" 

P*(T-t) matrix of quadratic form 
for performance index 

£*(T-t) weighting of state for 
optimal control 

$*(T-t+1,T-t) transition matrix 
under optimal regulation 

M(T-t) control transition matrix 

-Ai 

Q(T-t) matrix of quadratic form 
defining state error 

A 

R(T-t) matrix of quadratic form 
defining control effort 
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A. _ . .. sJ $.* . *! 


From Eq. (4.68), A*(t) may 


&*(t) 


00 • 


*OJ 


lijo 


• • • 


*JjJ 


ba expanded ns 

i 

_ “I • 


oo : 


• *■ 


l?jo • 7 • 


‘OJ 


•jjJ 


*0 




'»0 P 00 K 0 + U' 1 


$0J 


L^jo • • • 


^JoS lt Wo + « 


v \ v j^Woo% + R > 

k^O 


-1 


-1 


J J 
2. 

IfO k ! 


*>VWoo% 

u ■ • • 


•f K] 


J J 


«0 k « 0 ^ J ^ P ^ khI k IM 0 P 00 M 0 + 


-1 


-1 


r r > , ^ U i 'ilk 11 k^ M 0 P 00 M 0 + R] 


-1 


From Eq. (4.71), A*(t) may be expanded as 


(4.74) 


t*Ct} * [M*P 00 M 0 + R]-l ^ ■ . . 


Wo + *1-"*^! 

i JL-V-,- i 


i 


oo 


• • • 


OJ 


L p jo • • • p jjj 


^00 


• • • 


I J 

» 2 
i 

j k=^0 


• • ■ “5Vt 


00 


4 


^OJ 


l?J0 ' * 

. . . 4 


'JJJ 


OJ 


JO 


• •§■ • 


JJ 


f 
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J J 


J J 


I Wo + R)_1 « 


i-i / 


j j 


or . • A*Ct) - 1%P 00 M 0 + W-* 2 q 


(4.75) 


Application of Theorem 4.5 to Eq. (4.74) yields 


J J 


• k !o^ p ik^ [M i p oo M o + R'l 


1 1 *“1 


or, 


^Ct) 


J J -1 

Z Z [MlP A „M„ + R] XP,J. 


£«0 k~0 


0 00 0 




(4.76) 


From comparison of Eq. (4.75) and C4.76) it is obvious that Theorem 4.5 

can not. in general, be extended to the submatrix results. Since, in 

general, this is not true, then it can be concluded that the duality 

expressed by Theorem 4.5 does not apply to the corresponding submatrices. 

Attempts by the author to modify the duality theorem such that the re- 
» * 

suits may be extended to the submatrices have not been fruitful. If such 
e relationship c/nild be demonstrated then the results of Chapters 2 and 
3 should be the duals of each other. 

4.6 Summary and Conclusions 

In tills chapter an alternate expression for discrete systems 
with time delay has been developed. This expanded state representation 
is not new, yet it is shown to yield the same results as Chapter 2 and 


which used a different approach. These results are, then, submatrix 
solutions to the. expanded state representation. This method presents 
* substantial savings in computation time as well as storage, Examples 

* 

of these reductions are presented in Chapter 5. 

Finally* the expanded st^te representation allowed demonstration 

4 

of duality for time delay systems, although this result could not be ax- 

« . 

\ 

tended to the corresponding submatrices. 


• CHAPTER V 


COMPUTATIONAL ADVANTAGES OF PARTITIONED SOLUTIONS 

* 

TO OPTIMAL ESTIMATION AND CONTROL PROBLEMS 

/ 

* * * 

5.1 In. toduction 

Chapter 4 demonstrates that the filter and control gains obtained 
in this dissertation are Identical to results published many years ear- 
lier * using an expanded state representation of time delay systems 

The principal difference is that a partitioned form of the resulting al- 
gorithms are used rather than the expanded matrices. Since the expanded 

state representation is generally discredited because of the resulting 

» 

extensive computation required, one might ask whether or not the parti- 
tioned results represent a significant computational savings. 

Vj t 

In this chapter analytical expressions are developed for the 

* 

number of multiplicative and additive operations required for digital 
computer solution. Options are presented which reduce the number of op- 

• • t * 

erations even further in the partitioned form. These are at the expense 
of additional software requirements, however, and in some cases this more 
than offsets the advantages of the partitioned form. This is generally 

'4 

true where the system dimension and time delay are small. The savings 
in computation time are strong functions of the system order, n, and the 

t 

time delay magnitude, J. No further attempt is made here to define the 
additional software requirements for the partitioned solution since that 
depends to a great extent on the ingenuity of the programmer. 


Ill 


Soma illust rative examples are. presented, where computation time 

Is reduced by more- than 90% and storage by more than 60%. For a unit 

* 

delay, a 30% reduction in both, storage requirements and computation time 
is typical. 

/ . 

5.2 Computational Requirements lor Estimation Algorithms 

In this section a method of determining the number of discrete 
multiplicative and additive operations is presented. This method is 
then used to develop analytic expressions for the number of required 

operations in terms of the dimensions of the system matrices. Examples 

' » 

are presented at the conclusion of this section which demonstrate a sig- 

* ' » 

nificant reduction in computational requirements for the partitioned al- * 
gorithm when compared with the entire matrix algorithm of the expanded 
state form. 

Consider the multiplication of two matrices A and ,E of dimensions 
■ * 

l x m and m x n respectively. The number of discrete multiplicative op- 
erations is £mn. The number of discrete additive operations irequired is 
Umrl) n. For ease in calculations, it is assumed for the remainder of 
this chapter that the number of required additive operations is Iran. 

The number of required multiplicative and additive operations 
is determined for the partitioned algorithm expressed by Eqs. (2.115), 
(2.116) , (2.117) and (2.118). The dimensions of the matrices involved 
are: P(k+l,k+ljk) - n x n, <|> (k+l,k) - n x n, P(k-i,k-j|k) - n x n, 
P(k+l~i,k+l|k) - n x n, f(k+l,k) n x p* Q(k) - p x p, H(k+1) - m x n, 

- R(k+1) - m x m, K^(k) - n x m. 



.a 
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P(k.+l,fcH|k) « ,K £ 4> t (Jt+l , 10 P (M. , M | k) ^ ' (k+1 , k) 

1«0 JnO ^ 

+ r(fcH,k)Q(k)r'{k.+l,k) (5.1) 

J 

♦ ’ 

The. product <J>£(k+l,k)P(k-i,k.--j | k.) requires multiplicative operations. 

This product multiplied times <f> ' (k+l,k) requires an additional n” 5 multi- 

, •* 

plicative operations, resulting in 2n J multiplicative operations to form 
the product 

f^Ck+l , k.) P Ck-t , k-j | k) <f ’ (k+1 , k) ■ ' 

The double sum means there are (J+l) 4 such, terms, or, a total of 
2n^CJ+l)^ multiplications in the double sum. Similarly, the second term 
requires a total of n 2 p + p 2 n operations, yielding a total of 
2n^(J+l) A -!' n 2 p + p 2 n multiplicative operations. The number of additive 
operations are n 2 [(J+l) 2 - 13 for the double sum, plus n 2 by adding 
rCh+l,k)Q(k)f * (k+l,k) to the double sum, or a net of n 2 (J+l) 2 . To this 
must be added the number of additive operations due to the multiplica- 
tion process or 2n^(J+l) 2 4* n 2 p + p^n such additions. The total number 
of operations required in determining P(.k+l,k+l | k) . are 
multiplications: 2n^(J+l)^ + n 2 p + p 2 n 

additions: 2n^(J+l)^ + n^p + p 2 n + n^(J+l)^ 

A similar analysis yields the following results for the remaining equa- 

» 

tions of the partitioned estimation algorithm. 

K^k+1) = P (k+l-i , k+1 1 k) H ' (k+1) [ H (k+1 ) P (k+1 , k+1 1 k) H 1 (k+1 ) (5.2) 

+ RCk+l]" 1 i = 0, 1, ... , J (5.2) 


); 

({ 


i 



Total Number of Operations for 1-0,1, ... , J 
multiplications: n^mtJ+2] + 2nm^ 

additions: n 2 mlJ+2] + 2nm^ + 

J ~) 

P(k-j,k+l|k) - 1 P (k-j , k-i | k) <|) * (k+1 , k) j 

1=0 * * 


/' 

9 

0, 1, • • * » > J 
(5.3) 


Total Number of Operations for j =0, 1, ... , J 
* multiplications: n3j(j+l) 

additions; n 3 J(J+l) + n 2 J 2 

P(k-i,k-j|k) = P(k-i,k-j|k-l) - K.^ (k) H (k) P (k , k- j | k-1 ) (5.4) 

Total Number of Operations for i, j =0, 1, ... , J 
multiplications: n 2 ra[(J+l) 2 + (J+l)] 

additions: n 2 m[ (J+l) 2 + (J+l) ] + n 2 (J+l) 2 

The total number of' operations required for one complete iteration of 
the partitioned estimations algorithm are .given in Table 5.1. These 
results are referred to as PARTITION E-l. 

The number of required computations may be further reduced by 
recognizing that not all of the delayed states affect explicitly the 
one step transition. That is, some of, the operations are not required 
since some of the (^(k+ljk) = 0. Let j be the number of ^(k+l.k) which 
are not identically zero* Then it can be shown that the number of .com- 
putational operations becomes as shown in Table 5.1. Hereafter, these 
results are referred to as PARTITION E-2. 

Similarly, it can be shown, for the expanded algorithm given 
by Eqs. (5.5), (5.6) ’and (5,7) that the number of required operations 
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Tabla 5.1 

* 4 

Analytic Expression, of Number of Computational Operations 

Required for Various Estimation Algorithms 

* « 

/ 

* 

PARTITION E-l , 

multiplications; n 3 [2(J+l) 2 + j(j+l)] + n 2 m[(J+l) 2 + 2J + 3] 

* »■ . ‘ 

+ 2m 2 n + n 2 p 4- p 2 n 

additions; n 3 [2(J+l) 2 + JtmO] + n 2 m[(J+l) 2 + 2J + 3] 

+ 2m 2 n + n 2 p + p 2 n + m 2 + n 2 [2 (J+l) 2 + J 2 ] 

j * 

f i 

PARTITION E-2 

multiplications; n 3 [2j 2 + Jjj * n 2 m[2J + 3 + (J+l) 2 ] + 2m 2 n + n 2 p + p 2 n 
additions; n 3 [2j 2 + Jj] + n 2 ra[2J + 3 + (J+l) 2 ] + 2m 2 n + n 2 p + p 2 n 

+ n 2 [j 2 + J(j-l) + (J+l) 2 ] + m 2 

EXPANDED 

multiplications: 2n 3 (J+l) 3 + 4n 2 m(J+l) 2 + 2m 2 n 

■ » I | t ■» 

additions: 2n 2 (J+l) 2 + r 2 + 2n 3 (j+l) 3 +4n 2 m(J+l) 2 + 2m 2 n 


* a 
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Is as shown in Table 5,1. The piatrix dimensions are given in Table 5.3. 

• * t » « 

fc(k+l) - $ (k+1 | k)?!' (k+1) [ft (k+1)? (k+1 | k)ft(k+l) + feCk+1)]"' 1 (5.5) 

i i * * * 

ft (k+1 |k) ^ ^ (k+1 I k)ft (k | k)$ * (k+1 , k) + ft (k+1 , k) Q(k)ft ' (k+1 , k) (5 . 6) 

► * * t 

ft (k+1 1 k+1) ~ [I - ft (k+1 ) ft (k+1 ) J ft (k+1 \ k) (5.7) 

The effect of system order- (n) and time delay magnitude (J) on 
the required computational effort can be seen from. Table 5. 2^ where a 
number of different examples are considered. The data for PARTITION E-2 

m 

assumes, a single delay (j - 2) in the system, where the delay magnitude 

• 

is still specified by J. Note for the cases where there is just a unit 
delay (n - 2, J » 1) or (n - 10, J - 1) that the partitioned computations 

require only 60% of that required by the expanded form. For multiple 

• * » * 

delays, where the delay is longer (n ~ 1, J - 19) the computational re- 
quirements are less than 10% of those required by the expanded form. 

In general, the savings achieved are an increasing function of the time 
delay magnitude. 

5.3 Storage Requirements for Various Estimation Algorithms 

The storage requirements' are specified by the size of the ma- 
trices used in the algorithms. The partitioned algorithm is given by 
Eqs. (5.1) - (5.4) and the expanded state algorithm is given by Eqs. 

(5.5) - (5.7). The matrix dimensions and required storage are given in 
Table 5.3. Recall that PARTITION E-2 requires storage only for those 
values of $^(k+l,k) which are not null matrices. Storage requirements 
are computed for a number of examples in Table 5.4. A 30% reduction in 
required storage is achieved for a unit delay (J - 1) and a maximum 
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Table *5. 2 


Examples oi; Required Number op Computational Operations 
For Various Estimation Algorithms 


n 

J 

m 

■ p 

operation 

EXPANDED 

PARTITION 

E-l’PARTITION 

1 

3 

1 

l 

mult . 

200 

\ 

73 

45 





add. 

233 

115 

69 

2 

1 

1 

l 

mult. 

*200 

126 

126 





add. 

233 

* . 163 

163 

4 

0 

1 

l 

mult . 

200 

200 

>■ 

200 





add. 

233 

233 

•233 

1 

19 

i 

l 

mult. 

18060 

1625 

491 





add. 

18861 

2786 

915 

2 

9 

l 

l 

mult . 

18060 

2812 

702 




• 

add. 

18861 

3937 

1155 

4 

* 

4 

l 

l 

mult. 

18060 

5084 

1628 





add . 

18861 

6140 

2157 

10 

1 

l 

l 

mult. 

18060 

11030 

11030 





add . 

18861 

11931 

11931 

7 

5 

i 

• i 

mult. 

157087 

37457 

13790 





* add. 

161616 

* 41890 

16486 
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PARTITION E-2 - n [CJ+i) + (J+l) + j] -+ n[(J+2)m + P] + m + p 


savings of 60% can be seen from the analytical expressions for storage 

* 

requirements la Table 5.3. As In the preceding section, storage re- 
quirement savings Increase as the time delay magnitude Increases. 

t 

♦ 

5. A Computational Requirements for Control Algorithms 

\ 

The methods of Section 5.2 are' used in this section to deter- 
mine the computational requirements for the partitioned and expanded 
control algorithms. Results are obtained! for the partitioned algorithm 

under three conditions. First it is examined for the general case with 

> * 

all <{>£ not equal to zero. Next the computation is reduced by eliminating 
all operations for which <{>^ is equal to zero. Finally, it is shown that, 
at a nominal cost in additional storage requirements, a substantial re- 
duction in computation time may be achieved by storing the results of 

some operations which are repeated. As in the estimation problem, the 

• • 

partitioned algorithm represents a computational savings of 30% to 90% 

“ — . .... t . . 

* ♦ ... 

over the expanded form. * 

» 

PA RTITION C-l : The partition algorithm is expressed by Eqs. 

* 

(5.8) and (5.9). 

S(k) ■= [i(i , (k+l,k)W 00 (k+l)ip(k+l,k) + B(k)]"V (k+l,k) 

'• • ,[W 00 (k+i)^ i (k+l,k) + W 0>1+1 (k+l)] ' (5.8) 

W(k) « l4, i (k+l,k)W 00 (k+l) + W 1+lt0 (k+l)][fj(k+l,k) + i|j (k+l,k)S . (k) ] 

+ ^k+l,k)W 0ij+1 (k+l) + U i+1>j+1 (k+l) + A(k) 

(5.9) 

The matrix dimensions are given in Table 5.7. Application of the methods 
of Section 5.2 yield the number of required operations given in Table 5.5. 
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Table 5.4 

* » 
Examples of Required Storage 
For Various Estimation. Algorithms 


n 

J 

m 

P 

EXPANDED 

PARTITION E“1 

•'PARTITION E-2 

1 

3 

1 

1 

* 

62 

32 

30 

2 

1 

1 

1 

62 

44 

42 

4 

0 

1 

1 

62 

* .62 

62 

1 

m 

19 

1 

1 

1262 

464 

♦ 

446 

2 

9 

1 

1 

1262 

506 

474 

4 

4 

1 

1 

1262 

590 

542 

10 

1 

1 

1 

1262 

842 

842 

7 

5 

1 

1 

* 

5420 

2410 

2263 

(j«3) 





PARTITION C-2 : II the operations are eliminated lor those 

$£(k+l,k) which are equal to zero, the number of operations is reduced. 

II the number of <J>^(k+l,k) not equal to zero is j, the resulting number 

of computations can be shown to be those given in Table 5.5. 

* 

PARTITION C-3 : Examination of Eqs,. (5.8) end (5.9) reveals that 

* 

the term [WQQ(k+l)<j) ; ^(k+l,k) + Wq ^^(k+l)] occurs in the first equation 
and its transpose occurs in Eq. (5.9). This term may be stored at a 

4 

storage cost of n^(J+l) locations, these eliminating the necessity of 
re-computing a second time for each, iteration. The resultant computation 
requirements are given in Table 5.5. 

EXPANDED : The algorithm for the expanded form of control solu- 

tion is expressed by Eqs. (5.10) and (5.11). 

$(k) « [ $ ' (k+1 , k) (k+1 ) $ (k+1 , k ) + S(k),f ' 1 ^ t (k+l 1 k)W(k+l)$(k.-l*l,k) 

* 

*; (5.10) 

W(k) - J'(k+l,k)W(k)[J(k+l,k) + $(k+l,k)S<k) ) + A(k) (5.11) 
The matrix dimensions are given in Table 5.7. The required number of 
operations per iteration are given in Table 5.5. 

The effect of system order (n) and time delay magnitude (J) on 

the required computational .effort can be seen in Table 5.6 where a hum- •• 

> 

ber of different examples are considered. The data for PARTITION C-2 

and PARTITION C-3 assumes a single delay (j - 2) in the system where 

the delay magnitude is still expressed by J. For the cases where there 

is just a unit delay, (n =2, J « 1) and (n = 10, j = 1), the PARTITION 

% 

C-3 computations are only about 30% of those required for the expanded 
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Table 5.5 

* 

Analytic Expressions for the Number of Computational Operations 
Required for Various Control Algorithms / 

♦ 

PARTITION C-l 

♦ " 

multiplicatio ns : n^I2(J+l) 2 + J 4* 1] Hh n 2 r[l + (.J+l) + (J+l) 2 ] + 2nr 2 

additions: n 2 I2(J+l) 2 + J + 1] + n 2 r[l + J + 1 + (J+l) 2 ] + 

* •“ ' 1 u " * "• *■ 

+ 2nr 2 + r 2 + n 2 [J 2 + J + 2(J+I) 2 ] 

* 

PARTITION C-2 

multiplications : n 3 [(J+l) 2 + j(J+2)J + n 2 r[l + (J+l) + (J+i) 2 ] + 2nr 2 

additions : n 2 I(J+l) 2 + j(J+2)] + n 2 r[l + (J+l) + (J+l) 2 ] + 2nr 2 

+ r 2 + n 2 [ J 2 + 2Jj + 4j -J -2] 

PARTITION C-3 

multiplications ; n^[(J+l) 2 + j(J+l)3 + n 2 r[2J+3] + 2r 2 n 

additions ; n 3 [(J+l) 2 + j(J+l)] + n 2 r[2J+3] + ‘2r 2 n + r 2 | 

+ n 2 [J 2 + jJ + 2j] 

• ■' V 

additional storage required : n 2 ( J+l) 

EXPANDED 


multiplications : 3n^(J+l)? + 3n 2 r(J+l) 2 + 2n(J+l)r 

additions: 2n 2 (J+l) 2 + r 2 * + 3n 3 (J+l) 3 + 3n 2 r(J+l) 2 + 2n(j+l)r 
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Table 5.6 


Comparison. o£ Required Number of; Computational Operations 
For Various Control Algorithms 








j*=2 



N 

J 

r 

operation 

EXPANDED 

PARTITION PARTITION PARTITION 

EXTRA 






C-l 

C-2 

C-3 

STORAGE 

1 

3 

1 

mult. 

248' 

59 

47 

‘ 35 

4 




add. 

281 

103 

71 

55 


2 

1 

1 

mult. 

' 248 

112 

112 

88 

8 

m 



add. 

281 

153 

153 

117 


4 

0 

1 

mult . 

248 

248 

248 

184 

16 



\ 

add. 

281 

281 

281 

217 


1 

19 

1 

mult. 

25240 

1243 

863 

449 

20 




add . 

26041 

2420 

1288 * 

853 


2 

9 

1 

mult. 

25240 

2048 

1424 

1048 

40 




add . 

26041 

3209 

1881 

1453 


4 

4 

1 

mult . 

25240 

4016 

2872 

. 2424 

80 




add. 

26041 

5137 

3417 

2912 


10 

1 


/ mult. 

25240 

10720 

10720 

8520 

200 




add . 

26041 

11721 

11721 

9221 








j=3 

3=3 


7 

5 

i 

mult. 

227640 

28875 

20228 

19183 

294 




add. 

231169 

33874 

23169 

21437 



I 
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* * 

form. For multiple delays, where the delay is longer (n «* 1, J «* 19) 
the computational requirements of PARTITION C-l are* less than 5% of those 
required by the expanded form, in general, the savings achieved are an 
increasing function of the time delay magnitude. 

5.5 Storage Requirements for Various Control Algorithms 

The storage requirements are specified by the size of the ma- 

* 

trices used in the algorithms. The partitioned algorithm is given by 
Eqs. (5.8) and (5.9) and the expanded state algorithm is given by Eqs. 
(5.10 and (5.11). The matrix dimensions and required storage are given 
in Table 5.7. Recall that PARTITION C^2 requires storage only for those 
values of tj). (k-KL ,k) which are not null matrices. PARTITION C-3 requires . 
n^(J+l) storage in addition to that of PARTITION C-2 as explained in the 

preceding section. Storage requirements are computed for a number of 

> * * 

examples in. Table 5,8. A 30% reduction in required storage is achieved 

, v 

for a unit delay (J- 1) and a maximum savings' of 60% can be postulated 

\ 

from the analytical expressions for storage requirements in’ Table 5.7. 
Storage requirements decrease as the time delay magnitude increases. 

• 5.6 Summary 

As. demonstrated in, the preceding sections the partitioned algor- 
ithms obtained in this dissertation represent a 30% - 60% reduction in 
storage requirements when compared with, the expanded state representa- 
tion. A reduction in computational effort of 30% - 95% is also demon- 
strated. Even greater savings can be realized depending on the example 
selected. Both the reduction In computation time and storage make the 
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Table 5 . 8 

Examples of Required Storage 
for Various Control Algorithms 


XI 

J 

r 

EXPANDED 

PARTITION C-l 

/ 

PARTITION C-2 (j«2) 

1 

3 

1 

57 

26 

25 

2 

* 1 

1 

57 

35 

35 

4 

0 

1 

57 

57’ 

57 

1 

19 

1 

1241 

441 . 

425 

2 

9 

1 

1241 

449 

435 

4 

4 

1 

1241 

505 

473 

10 

1 

1 

1241 

731 

731 

7 

5 

1 

, 5377 

2010 

1961 (j=3) 



partitioned solutions quite attractive. In retrospect, the expanded 
state representation of time delay systems does provide algorithms which 
are computationally feasible after all. The need for a method to solve 

i 

* 

the expanded equations efficiently is resolved by this paper as the ex- 
amples of this chapter clearly demonstrate. It should be recalled, how- 
ever, that the partitioned solution has additional software requirements 
and this may offset the postulated advantages when J is small. 


CHAPTER VI: 


CONCLUSIONS AND FUTURE WORK 

* * 4 

6.1 Conclusions 

Ijn Chapters 2, 3 and A, algorithms are obtained for determining 

» * 

the optimal filter and optimal control gains fot discrete linear systems 
« * 
with, time delay. In Chapter A, it is demonstrated that these results 

are actually partitioned solutions to the expanded state representation 
of such systems. This expanded representation of time delay systems has 
been, studied but deemed computationally unacceptable by previous authors 
The unique nature of such a representation for time delay systems, how- 
ever, yields a substantial reduction in the computational requirements 
when the solution is expressed in partitioned form. It is not unlikely 
that the computations resulting from the partitioned solutions be 80% 
to 90% less than that formerly required by the expanded matrix solution. 
Many problems previously considered computationally unfeasible can as a 
consequence, now be solved. 

6.2 Future Work 

In the course of achieving these results, a number of related 
problems associated with time delay systems have arisen. 

1. What are the conditions under which continuous stochastic 
systems may be expressed as discrete systems? The initial conditions 
on the delay, since it is continuous, and the problem of delays which 
are not integral multiples of the sample period cause difficulty here. 

. ‘ 127 
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Soc\e preliminary vjork by the author suggests tha,t this may be possible 

*• 

for serial systems. 

2. Can controllability and observability as expressed in Appen- 

• ♦ 

dix A, be expressed in partitioned form? The achievement of significant 

results by partitioning the expanded representation of time delay systems 

suggests that this approach may be extended to other aspects of time de- 

lay systems. In Chapter 5 this effort failed in examining duality, how- 
*• 

ever, so there are some limitations on extending this approach. 

* • 

3. Are the results of this paper computationally superior to a 

high order approximation of the time delay? Since the required computa- 
tion increases as a significant function of the time delay magnitude, 
it may be more economical (and just as precise) to use a high order ap- 
proximation to the delay and then use standard techniques such as the 
Kalman filter for obtaining the final result. 

■4. .Can continuous algorithms be obtained by examining the lim- 
iting case of the discrete solutions? The continuous estimation and con- 

3 8 

trol solutions are known * for systems with time delay, but they are 
computationally untractable for high order systems (greater than n - 2) 
or more than a single unit delay Perhaps the solution to these contin- . 
uous-tirae equations may be determined by examining the limiting case of 
the discrete-time equations. This is certainly true for systems with no 
time delays. ; 




REQUIRED * PROOFS FOR CHAPTER II 


In this appendix necessary* end sufficient conditions are devel- 
oped for the observability and controllability of discrete linear systems 

* * 

With time delay. The development below: is for the expanded state repre- 
sentation of time delay systems. The reader is referred to Section 4.2 
for definitions of the matrices and vectors of* the system and measures 

ment equations (A.l) and (A. 2). 

* *■ f 

x(k+l) « (kll ,k)x(k) + $(k+l,k)uCk.) (A.l) 

* » 

. s!(k+l) - ft(k+l)x(k+l) (A. 2) 


Observability ; First, an observable system is defined as follows: 

Definition : The discrete linear ‘system of Eqs. (A.l) and 

* 

CA.2) is observable ,if x(0) can be determined from the 

. set of measurements ’ -$(1) , z(N)} for some finite 

N. If this is true for any initial time (k « 0 cor res- 

* 

ponds to tg) , the system is completely observable. 


The following theorem can now be proved. 

Theorem A.l : The discrete* linear system of Eqs. (A.l) 
and (A. 2) is completely observable if and only if the 
mN x n(J-fl) matrix 


XO) 

{tC2)?(2,l)^Cl J 0) 


^ ,n-d . . . la t o) 


(A. 3) 


130 





131 


is of rank n(J+l) for some N ? 0. 

» » t 

* 

* 

Proof: It ts sufficient to consider the case where u(k) *, 0. 

* * * 

x(k*KL) * $(k+X,k)x(k) • ' (A. 4) 

» * % 

z(k+l) » ft(k+l)x(k+l) ■ ,(A.5) 

► 

k. « 0, 1, . . . since fo(k) is assumed known for all k. 

Consider the sequence of measurements {z(l), ... , z(N)}. , In 
order for the matrix defined by Eq. (A. 3) to be of rank n(J+l), N is 
chosen such that the inequality mN 2s n(j+l) is satisfied. 

• From Eqs. (A. 4) and (A. 5) the following equations result from 
the sequence of measurements. 

*< l) » ftCDxd) = ftci)4(i,0)x(0) 

*< 2) = ft(2)i(2) = ft(2)?(2.1)ja,D)3E(0) 


z(N) « HCN)x(N) «*H(N)$(N,N-1) . . . J'(l f 0)xC0) 
These equations may, be written In matrix form as 


where 


z 


N 


?< N x(0) 


Zvr 2- z(l) 


CA.6) 


z(N) 


% 3 ftci)|a,0) 

ftC2)^C2,i)?a,o) 




ftCN)^CN,N-l) . . . $(1,0) 


(A.7) 


It’ is clear that“z N is an mN vector and x(0) is an n(J+l) vector. Since 


mN > n(J+l) and x(0) is arbitrary, the theory of linear equations may 


'( 

a 
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now be, Invoked . This theory states that there exists a unique solution 
to Eq. (A«6) if and only if the matrix as expressed by Eq. (A. 7 ), is 
of rank n 0+1) * Thus Theorem A. 1 is established. / 


Controllability ; First, controllability is defined in the following 

* 

sense. 


1. Definition A. 2 : The discrete linear system of 

Eq. (A.l) is controllable at time k « 0 (corres-^ 
ponding to an initial time tQ) if there exists 

i » 

a control sequence $ CO), u Cl) depend- 

* 

ing on ft(0) and the initial time, for which &(N) 
may be selected arbitrarily * where N is finite. 

If this is true for all x(0) and initial times, 
the system is completely controllable. 


The following theorem can now be proved. 

Theorem A. 2 ; The discrete linear system of 
Eq. (A.l) is completely controllable if and only 
if the n(J+l) x rN matrix 

[<KN,N-1)...$(2,1)<K1|0) .. .. f(N,N-l)^(N~l,N-2) ^(N,N-l)j 

(A*. 8) 


is of rank nfJ+1) for some N > 0 


Proof; The relationships for x(k) may be expressed in terms of x(C) and 
the control from Eq. (A.l) as 
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xCi) ** <>d,o)xCo) t ?(i,o)uCo) 

x(2) * f(2,l)?tl,0)xC0) + ?C2.1)?a,0)uC0) +?C2 l l)u(l) 


* » » * 

, x(N) * $(N,N~1) . . . $(l,0)x(0) 

t 2 

N i * 4 * 

+ z [ n <t.Cj,j-i)j^a,i-i)uCi-i) 

• i»l j=l 

* «• 

This sequence may be expressed in matrix form as 

» * 

&(N> - ? n 5SC0) * \\ 

* 

where x(N) is arbitrary 

* » » 

? H 3$(N,N-1) . . . $(2,l>?a,0) 


(A. 9) 


(A. 10) 


?„ £4>(N,N-l)...H2.1)'i'(l,0) ... (f(N,N-l)iKN-l,N-2) iKN.N-1) 


(A.l ) 


u N su(0) 


u(N-l) (A. 12) 
The definition requires that x(N) be arbitrary and it can be seen that 
x(N) is an n(J-KL) vector and is an rN vector where rN > n(J-fl) . 

Once again, Eq. (A. 9) has a unique solution, if and only if the matrix 
defined by (A. 11) is of rank n(J+l) and Theorem A. 2 is established. 


5 b 
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« APPENDIX B 

observability and controllability op time delay systems 

* » » 








OBSERVABILITY AND CONTROLLABILITY OF TIME DELAY SYSTEMS 

/ 

* * 

* * 

in this appendix. s properties (2.39) - (2.47) ar& established. 

* 

Recall the system and measurement equations for systems with, time delay. 

J 

x(k+l) ** t (k+l,k)x(k-i) + r(k+l,k)w(k) + ip (k+1 , k) u (k) (B.l) 
i*=0 

z(k+l) « H.(k+l)x(k+l) + v (k+1) * (B.2) 

These equations may also be imbedded in an expanded state representation 
x(k+l) ~ <jr(k+l,k)x(k) + ^(k+l,k)w(k) + !p(k+l,k)u(k) (B.3) 

’ z (k+1) « ?L(k+l)x(k+l) + v(k+l) (B.4) 

where the reader is referred to Section 4.2 for definitions of the ex- 
panded matrices and vectors. 

The following properties of the vectors are recalled for later 
use in the proofs below. 

{x(-i), 1=0,1,...,!} is a zero mean gaussian random n-vector 

(B.5) 

(w(k), k=0,l,...} is a zero-mean, p-dimensional gaussian white 
sequence (B.6) 

(v(k), k=l ,2 , . . .} •is a zero-mean, m-dimensional gaussian white 
sequence .... (B.7) 

(v ( j ) , j-1,2,...} and {w(k) , k-Q,l,...} are independent (B.8) 
{x(-i), i-0,l,,..,J} is independent of [v(k+l) , k=0,l,...} and 

t 

{w(k) , k=0 , 1 , . . . } . (B. 9) 

(u(k) , k=0,l,...} is known or may be specified as desired (B.10) 


it can be. shown that for a, system described by Eq. (B.3), the 

> t * » * 

state of; the' system at time k may be expressed in terms of the initial 

t * 

state, noise disturbance and control vector as / 


5c(k) « $Ch,0)x(0) + Z $(k,£)r0l,a,-l)w(4-l) 



+ • i $(k,0?(A»£^l)wCi-'l) 

£=1 

The partitioned expression for x(k) can then be written as 

J J k - 

xCk.) ■» Z ?0i^ k,0 ^ x ^“^ + E z Ci f f.**l)w(i-l) 

i«0 i=0 £=1 


CB.ll 


J k 

+ Z £ <J>i 0 CM)iKA,*-*l)uU-l) (B.12 
i“0 ^ 1 

where <J»^jCk,£) is the ijth. n x n submatrix of $(k,£). With the above 
results, the .properties of Chapter 2 can now be established. 


Property (2,39) : The stochastic processes {x(k), k=0,l,...} and 

{zCi), i"l,2,...,j} are gaussian’. 


(B. 13 


Proof : Consider Eq. (B.12). Recall that (u(j), j=0,l,,..} is a deter- 

ministic quantity by property (B. 10), -Since {x(-i), i=0,l,...,J} and 
each (w(£-l) , £=1,2 , . . „ ,k). are gaussian by hypothesis, it follows that 

x(k) is also gaussian for k - 0, 1, ... , since it is merely the sum of 

• * - * 

gaussian random vectors plus a deterministic vector. Consequently, for 
any integer m and set of time points {t^, t 2 » •••» t^el} the set ran 
dom n vectors x(ti), x(t 9 ), ..., x(t ) is jointly gaussian distributed 
and the assertion is proved. "" ' ' 
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Property (2.4Q) : E£x(j )w’ (k)} ^ 0 f or all k ^ 4 , j « 0* 1, ... (B.14) 

* • * * •» »% “ 

* - * ** ♦ * 

Proof : Eq. (JB.12) may be substituted ijato Eq'. (B.14) to yield 

9 

•U * . • . " ■ 

E{x(j)w'(k)} K £ (^>o i Ck. > 0)E^xC~i)w , (k)} 

i c 0 

+ b b <}> 1Q (j,i)ra,ii“i)E{wa-i)w f (k)> 
i-o jui 

j j 

• + £ £ nCj (i »i"'l)u(2,“l)E (w* (k)} (B.15) 

# , i*=0 i=l * . 

From property (B.9) the first term on the right-hand side of Eq. (B.15) 
vanishes for all k « 0, 1, ... . Since’ {w(k) , k-0,1,...} is a white se- 
quence. it follows that E {wC?,-l)w , (k)} *? 0 for all k ^ £-1* Since 
ij =; 1, 2, . . , , j it is clear that the second term vanishes for all 
k > j-1 or ki j. The third term vanishes since {w(k) , k«0 , 1 , • . . } has 

zero mean. Hence, Property (B.14) is established. 

. • ♦ 

Property (2.41) : E{z(j)w’(k)} = 0 for all- k ;> j , j « 0, 1, ... (B.16) 

Proof : Substitute Eq. (B.2) into Eq. (B.16) 

E{z(j)w , (k)} * H(j ) E {x (j )w* (k) } + E{v(j)w'(k)> (B.17) 

By virtue of Eq. (B.14) the first term on the right-hand side of Eq. 

(B.17) vanishes for all k £ j, j » 0, 1, ... . The second term vanishes 

by property (B.8) and the assertion (B.16) is established. 



Property (2.42) ; E£x(4)v’(k)} - 0 for all j * 0, 1, ... 

* i * 


and k « 1 , 2 , . , 


" (B.18) 

Proof; Substitute Eq, (B.12) into Eq. (B.16). 

' $ 

t 

3 „ 

Elx(j)v’Ck)} - t l 01 (i,0)E(x(-l)v'(k)} 

i*=0 


+ e e ^ n (.j^)ra,i-i)E{ W a-i)v'(k)} 

1=0 1=1 10 


+ E E $ (j,lHa,l-l)ua-l)E{v’(k)} (B.19) 
1=0 1 = 1 . lu - ’ 


The first term on the right-hand side vanishes by property (B.9). The 
second and third terms vanish by properties (B-8) and (B.7) respectively, 
and property CB.17) is established. 


Property (2. A3) ; E{z(j)v'(k)} - 0 for all k > j where j,k = 1, 2, ... 

‘ (B.20) 

Proof ; Substitute Eq. (B.2) into Eq. (B.18) 

E£z(j)v'(k)} - H ( j ) E {x ( j ) v * (k) } + E{v(j)v’(k)} (B.21) 

The first term on the right-hand side of Eq. (B.19) vanishes by property 
(B*17). The second term is equal to zero except for j ~ k since 
fv(k), k=l , 2 , . . . } is a white sequence. * But k > j by hypothesis and the 
assertion (B.20) is eatablished. 
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